Reformatting Statsbomb Data in Python

For this tutorial, I am going to carry on from where we finished in my first tutorial, which you can find that tutorial here. In that tutorial, we downloaded and installed the statsbombpy library and ran through the basic calls to download the free data released by Statsbomb.

In this tutorial, I am going to start with the basic call to get the match events from a single game, and start to parse out some of the information embeded within the file we receive from the call. It’s important to note, that the basic call in the Statsbomb library parses the JSON file in to a “tidy” dataframe. This means we are working with a Pandas dataframe and not with the raw JSON file. I will parse a raw JSON file in a future tutorial.

So let’s get started, first we need to import the libraries that we need to use.

# Read in appropriate libraries
from statsbombpy import sb # Statsbomb library to obtain data
import pandas as pd # Used to read in and manipulate data
import numpy as np # Used to help manipulate data

Once we have imported the libraries, we need to call the sb.events function to get the events from a single match. Once we do that, we need to have a look at the file, so we can do that by calling the head function on the first so many rows.

To view some of this data, you will need to scroll to the right of the tables presented below. This will apply to all tables in this blog and unfortunately was not something I could adjust.

### Run function to call the events from a single match
## Add match_id here
match = 2275038

## Run the event function using the assigned match from above
match_events = sb.events(match_id = match)
match_events.head(10)

credentials were not supplied. open data access only

So we have 41 columns of data, but we can’t see them all as Pandas will cut some of the columns out so as to not display too much information on the page. We can however print the column headers and see what we have to work with, or we can change some Pandas options to print the entire 41 columns for us. So lets change some options so we can also see what the data in these columns might look like.

### Change Pandas options to print max columns
pd.set_option('display.max_columns', None)

### Reprint head of data sorting by the minute column
match_events.sort_values('minute').head(10)

Great, now we can see all 41 columns and the data they contain. This should help us with rearranging and parsing out the data we need. Data for location for example provides an x / y value, as a list within the dataframe. This will need to be separated out before we could save this file, or use it effectively.

To do this, we are going to use a pd.Series. After reading a lot online, this is a slower method than using a Numpy tolist method, but handles NaN values much easier, which is something we are required to deal with in this dataset.

### First rename our dataframe
match_events_split = match_events

### Apply our split renaming columns for when we split the column
match_events_split[['location_x', 'location_y']] = match_events_split['location'].apply(pd.Series)

### Drop our location column as we don't need this anymore
match_events_split = match_events_split.drop('location', axis = 1)

### View the top of our file again to see this worked
match_events_split.head(10)

Now we have our locations parsed out, each of our event type columns, such as pass or shot, have a variety of information also contained within them. Such as if the pass was an assist, who received the pass and how long the pass was. All of this information is provided as a Numpy array within the dataframe. If we isolate the pass column and drop all NaN values, this is what we get.

### Select the pass column
pass_data_raw = match_events_split['pass']

### Drop NaN values from our selected column
pass_data_raw.dropna().head()

6     {'recipient': {'id': 10251, 'name': 'Fara Will...
7     {'recipient': {'id': 15725, 'name': 'Natasha H...
8     {'recipient': {'id': 18147, 'name': 'Kate Long...
9     {'recipient': {'id': 15725, 'name': 'Natasha H...
10    {'recipient': {'id': 22027, 'name': 'Anne Moor...
Name: pass, dtype: object

As we can see, there are lists within lists here and the information provided might be common across multiple event types within this dataset. Let’s see if we can pull anything further out of here and create a nice little dataframe of the information.

First, we can split the list in to a dataframe of values and rather than having our list of lists.

### Convert our list in to a dataframe
pass_data = pass_data_raw.apply(pd.Series)

### Filter our list to find our pass values
pass_data[pass_data.length >= 0]

Now we can see what our data actually includes, with a few of our columns still including a list of information., with ‘id’ and ‘name’ common within those columns, while we have a list of co-ordinates for our end locations as well. We can convert these columns one by one, but this would be time consuming to do for all individual variables in this dataset. For example, we can split each column like this:

### Split the height column in to separate columns
pass_data[['0', 'pass_height_id', 'pass_height_name']] = pass_data['height'].apply(pd.Series)

### Filter the dataframe to find our data
pass_data[pass_data.length >= 0]

Now that was relatively simple to do, but having to change the values for each column we want to split like this will take a fair amount of time to do. What we could do is write a function that checks each column of the dataframe and apply a function to it.

### Split pass data in to dataframe
pass_data_split = pass_data_raw.apply(pd.Series)

### Create function to loop through columns
### and apply a function to split the column
### in to id and name columns.
def pass_parse_function(data) -> pd.DataFrame:
    
    df = pd.DataFrame(data)
    dfcolumns = df.columns
    for i in dfcolumns:
        try: 
            df[['0', str(i) + '_id', str(i) + '_name']] = df[i].apply(pd.Series)
            df = df.drop(i, axis = 1)
        except ValueError:
            pass
    
    return df

### Run the function using the split dataframe
pass_df = pass_parse_function(pass_data_split)

### View the data from the function
pass_df[pass_df.length >= 0].head(10)

So there we have a function that parses out our columns and applies a new name to each colunmn. This is far simpler, and quicker than trying to write out a single line of code for each column we need to pull details from.

I’m not sure if this is the quickest function for doing this, so if anyone who uses Python is reading this, feel free to let me know if there is a better method of doing it.

Hopefully this tutorial provides you with some good information on using Python to look at Statsbomb data. I know there is a few things I can clean up in this process and hopefully I can learn / show you all those in the future.