Python split large list into sublists

In this tutorial, you will learn how to split a list into chunks in Python using different ways with examples.

Table of Contents Show

Method 1: Using a For-Loop
Method 2: Using the List Comprehension Method
Method 3: Using the itertools Method
Method 4: Using the NumPy Method
Method 5: Using the lambda Method
How to Split a List into n Sublists Using List Comprehension in Python
How to Split a List into Evenly Sized Chunks in Python
How to Split a List into n Sublists of Different Lengths in Python

Lists are mutable and heterogenous, meaning they can be changed and contain different data types. We can access the elements of the list using their index position.

There are five various ways to split a list into chunks.

Using a For-Loop
Using the List Comprehension Method
Using the itertools Method
Using the NumPy Method
Using the lambda Function

Method 1: Using a For-Loop

The naive way to split a list is using the for loop with help of range() function.

The range function would read range(0, 10, 2), meaning we would loop over the items 0,2,4,6,8.

We then index our list from i:i+chunk_size, meaning the first loop would be 0:2, then 2:4, and so on.

# Split a Python List into Chunks using For Loops sample_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] chunked_list = list() chunk_size = 2 for i in range(0, len(sample_list), chunk_size): chunked_list.append(sample_list[i:i+chunk_size]) print(chunked_list)

Output

[[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]]

Method 2: Using the List Comprehension Method

The list comprehension is an effective way to split a list into chunks when compared to for loop, and it’s more readable.

We have a sample_list and contain ten elements in it. We will split the list into equal parts with a chunk_size of 2.

# Split a Python List into Chunks using list comprehensions sample_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] chunk_size=2 result=[sample_list[i:i + chunk_size] for i in range(0, len(sample_list), chunk_size)] print(result)

Output

[[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]]

sample_list[i:i + chunk_size] give us each chunk. For example, if i=0, the items included in the chunk are i to i+chunk_size, which is from 0:2 index. In the next iteration, the items included would be 2:4 index and so on.

Method 3: Using the itertools Method

We can leverage the itertools module to split a list into chunks. The zip_longest() function returns a generator that must be iterated using for loop. It’s a straightforward implementation and returns a list of tuples, as shown below.

# Split a Python List into Chunks using itertools from itertools import zip_longest sample_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] chunk_size=2 result=list(zip_longest(*[iter(sample_list)]*chunk_size, fillvalue='')) print(result) chunked_list = [list(item) for item in list(zip_longest(*[iter(sample_list)]*chunk_size, fillvalue=''))] print(chunked_list)

Output

[[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]]

Method 4: Using the NumPy Method

We can use the NumPy library to divide the list into n-sized chunks. The array_split() function splits the list into sublists of specific size defined as n.

# Split a Python List into Chunks using numpy import numpy as np # define array and chunk_szie sample_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] our_array = np.array(sample_list) chunk_size = 2 # split the array into chunks chunked_arrays = np.array_split(our_array, len(sample_list) / chunk_size) print(chunked_arrays) # Covert chunked array into list chunked_list = [list(array) for array in chunked_arrays] print(chunked_list)

Output

[array([1, 2]), array([3, 4]), array([5, 6]), array([7, 8]), array([ 9, 10])] [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]]

Method 5: Using the lambda Method

We can use the lambda function to divide the list into chunks. The lambda function will iterate over the elements in the list and divide them into N-Sized chunks, as shown below.

# Split a Python List into Chunks using lambda function sample_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] chunk_size = 2 lst= lambda sample_list, chunk_size: [sample_list[i:i+chunk_size] for i in range(0, len(sample_list), chunk_size)] result=lst(sample_list, chunk_size) print(result)

Output

[[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]]

In Python, we can split a list into n sublists a number of different ways. Given a length, or lengths, of the sublists, we can use a loop, or list comprehension to split a list into n sublists.

list_of_numbers = [0,1,2,3,4,5] def getSublists(lst,n): subListLength = len(lst) // n list_of_sublists = [] for i in range(0, len(lst), subListLength): list_of_sublists.append(lst[i:i+subListLength]) return list_of_sublists print(getSublists(list_of_numbers,3)) #Output: [[0, 1], [2, 3], [4, 5]]

A more efficient way to split a list into sublists is with yield().

list_of_numbers = [0,1,2,3,4,5] def getSublists(lst,n): subListLength = len(lst) // n for i in range(0, len(lst), subListLength): yield lst[i:i+subListLength] print(list(getSublists(list_of_numbers,3))) #Output: [[0, 1], [2, 3], [4, 5]]

Here is how to split a list into n sublists using list comprehension in Python.

list_of_numbers = [0,1,2,3,4,5] def getSublists(lst,n): subListLength = len(lst) // n return [lst[i:i + subListLength] for i in range(0, len(lst), subListLength)] print(getSublists(list_of_numbers,3)) #Output: [[0, 1], [2, 3], [4, 5]]

When working with lists in Python, the ability to manipulate them and create new lists is very valuable.

One such manipulation is the ability to split a list into n sublists in Python.

We can use a loop to break a list into n sublists.

To split a list into n sublists, first we need to calculate the length of each sublist. While it might not always work out that there are equal lengths for all n sublists, we can get pretty close.

After getting the length of each sublist, we can then loop over the list and create a list of lists with slicing.

Below is a Python function which will split a list into n sublists with a for loop.

Below shows what will happen if you cannot divide the length of the list by the number of sublists equally.

list_of_numbers = [0,1,2,3,4,5,6] def getSublists(lst,n): subListLength = len(lst) // n list_of_sublists = [] for i in range(0, len(lst), subListLength): list_of_sublists.append(lst[i:i+subListLength]) return list_of_sublists print(getSublists(list_of_numbers,3)) #Output: [[0, 1], [2, 3], [4, 5], [6]]

A more efficient way to split a list into sublists is with yield().

How to Split a List into n Sublists Using List Comprehension in Python

We can also use list comprehension to split a list into n sublists using Python.

To split a list into n sublists, first we need to calculate the length of each sublist. While it might not always work out that there are equal lengths for all n sublists, we can get pretty close.

After getting the length of each sublist, we use list comprehension to loop over the list and creating a list of lists.

Below is a Python function which will split a list into n sublists with list comprehension.

How to Split a List into Evenly Sized Chunks in Python

We can easily modify our function from above and split a list into evenly sized chunks using Python. Instead of calculating the chunk size in the function, we accept it as an argument.

Below is how to split a list into evenly sized chunks using Python.

list_of_numbers = [0,1,2,3,4,5] def getSublists(lst,chunkSize): for i in range(0, len(lst), chunkSize): yield lst[i,i + chunkSize] print(list(getSublists(list_of_numbers,2))) #Output: [[0, 1], [2, 3], [4, 5]]

How to Split a List into n Sublists of Different Lengths in Python

The first two examples in this article showed us how to split a list into n sublists where each of the n sublists had an equal, or almost equal length.

You can split a list into n sublist with different lengths with the help of the islice() function from the itertools module.

We can create a function which takes in a list and a list of the lengths you want each sublist to have, and return a list of sublists.

Below is a Python function which will split a list into sublists of given lengths.

from itertools import islice list_of_numbers = [0,1,2,3,4,5,6,7,8,9,10] lengths = [3,2,4,2] def getSublists(lst,lens): iter_lst = iter(lst) return [list(islice(iter_lst, x)) for x in lens] print(getSublists(list_of_numbers,lengths)) #Output: [[0, 1, 2], [3, 4], [5, 6, 7, 8], [9, 10]]

Hopefully this article has been useful for you to learn how to split a list into n sublists using Python.