Advanced JSON code samples in Python

JSON is one of the most widely used format to transfer data in the form of key-value pairs or arrays. JSON being light-weight and language-independent made it widely accepted form of data transfer. That is why, almost all of the programming languages including Python provides built-in tools to parse (read or modify) JSON data.

Some points to remember regarding JSON

Data can be stored as arrays or key-value pairs or a combination (ie. Values of a key-value pair can be an array or a group of key-value to form an array, etc).

[] to represent arrays as a collection of items.

{} to represent key-value pairs as objects of a class.

Comma to separate each part of data.

Keys and string values are represented within quotes. Number, boolean and null values represented without quotes.

Python and JSON

Python provides a built-in standard library to work with the JSON data. To use this, we just need to import the library using the the code import json and use loads() method to read the JSON data and dumps() to convert any serializable object to JSON.

These are the only two methods that we really need from JSON library when we work with JSON data. Rest of the methods that we use are mainly associated to the particular data structures. This will be clear as you go through the below samples.
Check the date structures and its methods here.
In Python, data types strings, int, float, bool and data structures list, tuple, dict are serializable and can be converted to JSON directly.

Working with JSON is pretty simple and straight forward in Python, in the case of simpler or smaller set of data, be it a set of key-value pairs or a list of items. But there may often be situations where we will have to work with bigger and complex JSON strings.

As mentioned above, data structures list, tuple, dict are serializable. List and tuple stores data as a collection of items and dict stores data as key-value pairs which can be used to represent data as objects.

As we know, lists or tuples can be 2 dimensional or multi-dimensional. ie, A group of lists can be joined together to form another list and the lists within can be of variable lengths as well. Complexities arise when we try to filter or modify lists within lists (can be 2D lists or 3D lists or more). Further more, single or multi-dimensional lists can be used as the value of key-value pairs of a dictionary.

Here in this topic, we are trying to cover all the above said scenarios with examples, starting with the simplest use cases. We will see how to read and filter complex JSON strings and also to update particular parts of the JSON data.

Read data from a JSON array

To read a particular data from a simple JSON array string, we first have to parse and read the JSON array using loads() method and then use the positional values as shown below. As you can see in the example, loads() method converts JSON array to a Python list and then we can use all the methods available for List data structure to read the data.
Copied

import json jsonStr = '["Car", "Bus", "Bike", 1, 2]' lst = json.loads(jsonStr) print(lst) print("Length : ", len(lst)) print("Item at third position : ", lst[2]) ##Looping through items for l in lst: print(l)
Output: ['Car', 'Bus', 'Bike', 1, 2] Length : 5 Item at third position : Bike
Write to a JSON array

In the above example we are reading from JSON array string. If we want to modify a particular value of the the JSON array, we have to convert this to a List using loads() method, then update the value in the required position and then convert it back to JSON string using dumps() method.
Copied

import json jsonStr = '["Car", "Bus", "Bike", 1, 2]' lst = json.loads(jsonStr) lst[2] = "Motor Bike" newjsonStr = json.dumps(lst) print("Updated JSON String : ", newjsonStr)
Output: Updated JSON String : ["Car", "Bus", "Motor Bike", 1, 2]
Read and write data to a JSON string in key-value format

In this sample, we are considering a collection of key value-pairs in the simplest of formats. Consider this a class with one property. Example shows parsing json string to a python dictionary collection.

From this dictionary, we are fetching an item, updating an item and also filtering a part of the collection and applying changes to it (using List Comprehension). After this the dictionary is converted back to JSON.
Copied

import json jsonStr = '{ "n1":"James", "n2":"Rory", "n3":"Kevin"}' dict1 = json.loads(jsonStr) print("Number of employees : ", len(dict1)) print("Name of first Employee : ", dict1["n1"]) dict1["n3"] = 'Kev' newjsonStr = json.dumps(dict1) print("Name changed for third employee : ", newjsonStr) updateddict = { key:val + ' SS' for key, val in dict1.items() if (val == 'James' or val == 'Rory')} newjsonStr = json.dumps(updateddict) print("Filter some employees and update name : ", newjsonStr)
Output: Number of employees : 3 Name of first Employee : James Name changed for third employee : {"n1": "James", "n2": "Rory", "n3": "Kev"} Name changed for James and Rory : {"n1": "James SS", "n2": "Rory SS"}
Read and write a JSON string representing a class with multiple properties and lists within

Consider a student list with id, name and marks for multiple subjects. This is an example of a JSON array with multiple key-value pairs (objects) within. This can be represented in JSON as an array of key-value pairs, with marks as another key-value pair within, as shown in below sample.
```
[
	{"id": 1, "name": "Carl",
		"marks": {"english": 48, "science": 47, "maths": 45}},
	{"id": 2, "name": "Tim",
		"marks": {"english": 43, "science": 42, "maths": 41}},
	{"id": 3, "name": "Peter",
		"marks": {"english": 40, "science": 44, "maths": 47}}
]
```
If we represent this as a class, it will be like;
```
class student
	int id
	string name
	list<marks>

class marks
	int english
	int science
	int maths
```
This may be one of the most common scenarios that we can expect while working with JSON data. We will see how to read a particular data from the collection, update a single data or a group of data at once. To update multiple items at once in such kind of lists, we can use another python library operator.
Copied

import json import operator students = ' [{"id": 1, "name": "Carl", "marks": {"english": 48, "science": 47, "maths": 45}}, {"id": 2, "name": "Tim", "marks": {"english": 43, "science": 42, "maths": 41}}, {"id": 3, "name": "Peter", "marks": {"english": 40, "science": 44, "maths": 47}}]' studentsList = json.loads(students) print("Marks of First Student : ", studentsList[0]["marks"]) studentsList[0]["marks"]["science"] = 37 newjsonStr = json.dumps(studentsList) print("After updating mark of First Student in Science: ", newjsonStr) [operator.setitem(st, "total", st['marks']['english'] + st['marks']['science'] + st['marks']['maths'] ) for st in studentsList] print("Finding total marks of each student: ", json.dumps(studentsList))
Output: Marks of First Student : {'english': 48, 'science': 47, 'maths': 45} After updating mark of First Student in Science: [{"id": 1, "name": "Carl", "marks": {"english": 48, "science": 37, "maths": 45}}, {"id": 2, "name": "Tim", "marks": {"english": 43, "science": 42, "maths": 41}}, {"id": 3, "name": "Peter", "marks": {"english": 40, "science": 44, "maths": 47}}] Finding total marks of each student: [{"id": 1, "name": "Carl", "marks": {"english": 48, "science": 37, "maths": 45}, "total": 130}, {"id": 2, "name": "Tim", "marks": {"english": 43, "science": 42, "maths": 41}, "total": 126}, {"id": 3, "name": "Peter", "marks": {"english": 40, "science": 44, "maths": 47}, "total": 131}]
Read and write a JSON string representing a list of lists

The above example can be considered as objects of a class. Now let us consider if the same set of data is not represented in object format. ie, Data will not be represented as key-value pairs, but as a three-dimensional (3D) array as a list of items within another list all grouped in another list.
```
[
    [1, 'Carl', 
        [48, 47, 45]
    ],
    [2, 'Tim', 
        [43, 42, 41]
    ],
    [3, 'Peter', 
        [40, 44, 47]
    ]
]
```
Let us check how to work with a 3 dimensional JSON array. Read and update of a single item are straight forward. To update multiple items at once, we are using List Comprehension and adding a forth column to find the total.
Copied

import json students = '[ [1, "Carl", [48, 47, 45] ], [2, "Tim", [43, 42, 41]], [3, "Peter", [40, 44, 47]] ]' studentsList = json.loads(students) print("Marks of First Student : ", studentsList[0][2]) studentsList[0][2][1] = 37 newjsonStr = json.dumps(studentsList) print("After updating mark of First Student in second subject: ", newjsonStr) totalMarks = [ [ st[0], st[1], st[2], st[2][0] + st[2][1] + st[2][2] ] for st in studentsList] print("Finding total marks of each student: ", json.dumps(totalMarks))
Output: Marks of First Student : [48, 47, 45] After updating mark of First Student in second subject: [[1, "Carl", [48, 37, 45]], [2, "Tim", [43, 42, 41]], [3, "Peter", [40, 44, 47]]] Finding total marks of each student: [[1, "Carl", [48, 37, 45], 130], [2, "Tim", [43, 42, 41], 126], [3, "Peter", [40, 44, 47], 131]]

Conclusion

Any serializable objects like strings, int, float, bool, list, tuple or dict can be parsed to JSON using python.

Use loads() method to convert JSON string to list or dictionary and then work on this list to modify the data and use dumps() method to convert it back to JSON.

To read or update a particular value within JSON string we can use positional values or any available data structure methods.

To bulk update or filter, it is easier to use List Comprehension.

Advanced JSON code samples in Python

Some points to remember regarding JSON

Python and JSON

Read data from a JSON array

Write to a JSON array

Read and write data to a JSON string in key-value format

Read and write a JSON string representing a class with multiple properties and lists within

Read and write a JSON string representing a list of lists

Conclusion