Table of Contents
Hello friend! Arrays are essential in Python programming – let‘s explore them.
I will walk you through all key aspects of 2D arrays step-by-step. By the end, you will have mastered:
- Array creation, manipulation and processing
- Cutting edge techniques like vectorization
- Applying arrays to tackle real-world problems
So buckle up for this action-packed guide!
Why Are 2D Arrays Useful?
2D arrays (also called matrices) help store data in tabular format – like spreadsheets with rows and columns.
This data structure is ubiquitous:
- ~80% of real-world data is tabular – finance/weather records, image pixels, medical stats
-~60% of Python developers use NumPy arrays to process data according to 2021 StackOverflow survey
No wonder 2D arrays lie at the heart of:
- Machine learning models
- Data analytics pipelines
- Scientific computing tasks
Having the ability to leverage 2D data can set you apart as a Python programmer.
You gain skills to:
- Store and query tabular data
- Manipulate matrices with ease
- Process signals/images efficiently
- Use libraries like Pandas, TensorFlow
Now let‘s see how to create arrays!
Initializing 2D Arrays
There are popular ways to define arrays in Python:
- Lists – great flexibility
- NumPy arrays – performance + functions
- Custom arrays – type safety
Let‘s compare them hands-on with code examples.
Lists
Lists offer flexibility by allowing mixed data types and dynamic sizes:
matrix = [[1, "Mark", 3.5],
[4, 5, 6.5]]- Create any number of rows and columns quickly
- Nested lists act like matrices
- Append new rows/columns easily
However, lists have limitations:
- No vectorized operations for performance
- Higher memory usage
Still, lists work well for smaller data and rapid prototyping.
NumPy NDArrays
The NumPy library provides multi-dimensional arrays for fast math/stats operations.
Let‘s import NumPy and create an array:
import numpy as np
matrix = np.array([[1, 2, 3],
[4, 5, 6]]) We passed a list of lists to np.array(). NumPy hands us back a fast NDArray with benefits like:
- Speed – highly optimized C backend
- Vectorization – apply functions to entire arrays
- Broadcasting – propagate scalar values
- Methods – math/stat/logic capabilities
Behind the scenes, NumPy stores data in a contiguous block rather than references. This results in less memory usage and faster access.
No wonder NumPy is used for scientific computing!
While NumPy arrays lack flexibility, they enable high performance numeric programming.
Custom Arrays
For specialized use cases like embedded systems, you may want arrays with fixed data types or memory layouts.
The built-in array.array type helps with this by allowing us to specify the data type code:
from array import *
matrix = array(‘f‘, [[1.1, 1.2],
[1.3, 1.4]]) We passed the float type code ‘f‘ here. Other options are ‘i‘ for integer or ‘u‘ for unsigned integer.
The array module is great when you need:
- Arrays limited to one data type
- Control over memory allocation
- Interaction with C/C++ code
So in summary, we looked at various array initialization options in Python that cater to different scenarios.
Up next, let‘s slice and dice arrays!
Indexing and Slicing Arrays
We often need to extract partial data from arrays. Instead of looping through each element, we can leverage indexing and slicing.
Let‘s start with a NumPy array:
import numpy as np
arr = np.array([[1, 2, 3, 4],
[5, 6, 7, 8]])Indexing
We pass row and column indexes within square brackets to access elements. Counting starts at 0.
For example, fetch the number 6:
num = arr[1, 1] # row 1, col 1
print(num)
# 6We passed [row, column] style indexes inside square brackets.
We can also pass a single index to get entire rows/columns:
row = arr[1] # 2nd row
col = arr[:, 1] # 2nd column
print(row)
# [5 6 7 8]
print(col)
# [2 6] The comma separated syntax arr[:, 1] returns the slice across all rows.
Slicing
What if we want to fetch a range of elements instead of individual ones?
Slicing comes to the rescue by allowing us to extract sections of an array:
subset = arr[:2, 2:] # first 2 rows, cols 2 till end
print(subset)
# [[3 4]
# [7 8]]The : operator lets us specify a range to slice. This returns a view on to the same data buffer.
- Positive indexes start counting from the first element
- Negative indexes count backwards from the last element
Together, indexing + slicing provide flexible ways to access array data without using slow loops!
Modifying Array Elements
Let‘s see how to modify existing arrays.
We can directly change elements with indexing:
arr[0, 0] = 99 # Update 1 to 99
print(arr)
# [[99 2 3 4]
# [ 5 6 7 8]] To insert new elements, use np.insert():
arr = np.insert(arr, 1, 55, axis=1)
print(arr)
# [[99 55 2 3 4]
# [ 5 55 6 7 8]]Set axis=1 to insert along column at index 1.
We can append new rows or columns to arrays as well:
row = [[9, 9, 9, 9]]
arr = np.append(arr, row, axis=0)
print(arr)
# [[99 55 2 3 4]
# [ 5 55 6 7 8]
# [9 9 9 9]]Again, pick axis=0 to stack vertically.
There are similar techniques to delete or filter array elements. The main takeaway is that arrays enable easy element manipulations.
Broadcasting in NumPy
What if we want to add a fixed value to every array element?
Sure, we can loop through and update each cell – but that becomes slow for large arrays.
Broadcasting is a faster vectorized approach.
The idea is that NumPy expands smaller arrays automatically during operations to match larger array shapes.
Let‘s see it in action:
import numpy as np
arr = np.array([[1, 2],
[3, 4]])
print(arr + 5)
# [[6 7]
# [8 9]]Here NumPy extended the scalar 5 into a same-sized array and added element-wise. Broadcasted addition!
We can use broadcasting to vectorize any NumPy operation efficiently including:
- Math functions –
sin,log - Comparisons –
>,<= - Aggregations –
sum,max
It saves us explicit loops and unlocks speed through vectorization.
Up next, let‘s explore iterating through arrays.
Iterating Over Arrays
In reality, broadcasting cannot solve every problem. We often need to write explicit loops to process array data.
Let‘s revisit our array:
arr = np.array([[1, 2, 3],
[4, 5, 6]])For Loops
The classic way is to iterate through each row manually:
for row in arr:
print(row) # Prints rows
# [1 2 3]
# [4 5 6]We can access both the row and each column value inside the loop:
for row in arr:
for col in row:
print(col) # Prints values
# 1
# 2
# 3
# 4
# 5
# 6 Nested loops help process multidimensional data.
While simple, regular Python loops can get slow for huge arrays. So let‘s look at alternatives.
Vectorized Functions
NumPy universal functions are fast element-wise vectorized versions of common math functions:
arr = np.array([1, 2, 3, 4])
doubles = np.multiply(arr, 2)
print(doubles)
# [ 2 4 6 8] Here np.multiply() has internally broadcasted and doubled each number – no loop required!
Modern hardware accelerates such vectorized functions using:
- SIMD – Single instruction, multiple data
- GPUs – Massively parallel processors
- Vector engines – Pipelining computations
Vectorization thus unlocks order-of-magnitude speedups compared to explicit loops in Python.
nditer objects
np.nditer provides fine-grained control during iteration. It works well for unusual data layouts.
We can specify ordering and chunking behavior to optimize cache usage:
for x in np.nditer(arr, order=‘C‘):
print(x) The order=‘C‘ parameter processes data in row-major order to maximize locality.
So in summary, array iterations in NumPy scale from simple to advanced techniques.
Case Study: Image Processing
Now that we have covered arrays in depth, let‘s apply them to a real-world problem – image processing.
Images are just 2D arrays of pixel values. Various image operations boil down to array transformations.
Let‘s load, manipulate and save an image:
from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
img = Image.open(‘image.jpg‘)
arr = np.asarray(img) # Convert image to array
print(arr.shape)
# (200, 400, 3)
arr_grey = np.sum(arr, axis=2) / 3 # Make grayscale
img_grey = Image.fromarray(arr_grey)
img_grey.save(‘image_grey.jpg‘)We opened the RGB image into a Numpy array, computed a grayscale version by averaging pixel channels before saving back.
Here are some common image processing operations powered by array programming:
- Change contrast/brightness
- Create image filters – blur, sharpen
- Face detection using computer vision
Many machine learning tasks rely on optimized image data pipelines.
Let‘s wrap up with best practices.
Key Takeaways
We covered a lot of ground on 2D arrays. Let‘s summarize the key lessons:
- Initialize arrays using lists or NumPy based on flexibility vs performance needs
- Index & slice arrays to conveniently access elements
- Modify arrays with insert, append, delete methods
- Vectorize operations using broadcasting to avoid slow loops
- Apply multi-dimensional data analysis to real-world problems like images
Arrays will boost your ability to wrangle and glean insights from data.
I hope you enjoyed reading this guide as much as I liked writing it! I welcome your thoughts/feedback to improve future versions for other learners.