Some of the most useful Pandas tricks

Image for post
Image for post
All Pandas json_normalize() you should know for flattening JSON (Image by Author using canva.com)

Reading data is the first step in any data science project. As a machine learning practitioner or a data scientist, you would have surely come across JSON (JavaScript Object Notation) data. JSON is a widely used format for storing and exchanging data. For example, NoSQL database like MongoDB store the data in JSON format, and REST API’s responses are mostly available in JSON.

Although this format works well for storing and exchanging data, it needs to be converted into a tabular form for further analysis. You are likely to deal with 2 types of JSON structure, a JSON object or…


Image for post
Image for post
All Pandas cut() you should know for transforming numerical data into categorical data (Image by author using canva.com)

Numerical data is common in data analysis. Often you have numerical data that is continuous, or very large scales, or is highly skewed. Sometimes, it can be easier to bin values into discrete intervals. This is helpful to perform descriptive statistics when values are divided into meaningful categories. For example, we can divide the age into Toddler, Child, Adult, and Elder.

Pandas’ built-in cut() function is a great way to transform numerical data into categorical data. In this article, you’ll learn how to use it to deal with the following common tasks.

  1. Discretizing into equal-sized bins
  2. Adding custom bins
  3. Adding…


All you need to know about Pandas Series — the basic building blocks of a DataFrame.

Image for post
Image for post
A practical introduction to Pandas Series (Image by Author using canva.com)

DataFrame and Series are two core data structures in Pandas. DataFrame is a 2-dimensional labeled data with rows and columns. It is like a spreadsheet or SQL table. Series is a 1-dimensional labeled array. It is sort of like a more powerful version of the Python list. Understanding Series is very important, not only because it is one of the core data structures, but also because it is the building blocks of a DataFrame.

In this article, you’ll learn the most commonly used data operations with Pandas Series and should help you get started with Pandas. …


Getting Started

A practical introduction to ReLU with Keras and TensorFlow 2

Image for post
Image for post
Why ReLU in Deep Learning (image by author using canva.com)

The activation functions are at the very core of Deep Learning. They determine the output of a model, its accuracy, and computational efficiency. In some cases, activation functions have a major effect on the model’s ability to converge and the convergence speed.

In this article, you’ll learn why ReLU is used in Deep Learning and the best practice to use it with Keras and TensorFlow 2.

  1. Problems with Sigmoid and Tanh activation functions
  2. What is Rectified Linear Unit (ReLU)
  3. Training a deep neural network using ReLU
  4. Best practice to use ReLU with He initialization
  5. Comparing to models with Sigmoid and…


TensorFlow 2 tutorials

A practical introduction to Sigmoid, Tanh, ReLU, Leaky ReLU, PReLU, ELU, and SELU

Image for post
Image for post
7 popular activation functions in Deep Learning (Image by author using canva.com)

In artificial neural networks (ANNs), the activation function is a mathematical “gate” in between the input feeding the current neuron and its output going to the next layer [1].

The activation functions are at the very core of Deep Learning. They determine the output of a model, its accuracy, and computational efficiency. In some cases, activation functions have a major effect on the model’s ability to converge and the convergence speed.

In this article, you’ll learn the following most popular activation functions in Deep Learning and how to use them with Keras and TensorFlow 2.

  1. Sigmoid (Logistic)
  2. Hyperbolic Tangent (Tanh)


A practical introduction to the custom callback

Image for post
Image for post
Image made by author using www.canva.com

Callbacks are an important type of object in Keras and TensorFlow. They are designed to be able to monitor the model performance in metrics at certain points in the training run and perform some actions that might depend on those performances in metric values.

Keras has provided a number of built-in callbacks, for example, EarlyStopping, CSVLogger, ModelCheckpoint, LearningRateScheduler etc. Apart from these popular built-in callbacks, there is a base class called Callback which allows us to create our own callbacks and perform some custom actions. …


Some of the most useful Pandas tricks

Image for post
Image for post
converting JSON into a Pandas DataFrame (Image by Author using canva.com)

Reading data is the first step in any data science project. Often, you’ll work with data in JSON format and run into problems at the very beginning. In this article, you’ll learn how to use the Pandas built-in functions read_json() and json_normalize() to deal with the following common problems:

  1. Reading simple JSON from a local file
  2. Reading simple JSON from a URL
  3. Flattening nested list from JSON object
  4. Flattening nested list and dict from JSON object
  5. Extracting a value from deeply nested JSON

Please check out Notebook for the source code.

1. Reading simple JSON from a local file

Let’s begin with a simple example.

[ { "id"…


A step by step tutorial for scraping tables from a JavaScript webpage

Image for post
Image for post
Scraping tables from a JavaScript webpage using Selenium, BeautifulSoup, and Pandas (Image by author using canva.com)

Web scraping is the process of collecting and parsing data from the web. The Python community has come up with some pretty powerful web scrapping tools. However, many modern websites are dynamic, in which the content is loaded and populated using client JavaScript. Therefore, some extra setups are required in order to scrape data from JavaScript webpages.

In this article, you’ll learn how to scrape tables from a JavaScript webpage using Selenium, BeautifulSoup, and Pandas.

  1. Challenges of a JavaScript webpage
  2. Install libraries and Selenium web driver
  3. Scrap tables using Selenium, BeautifulSoup, and Pandas

Please check out the source code from…


Some Pandas read_html() tricks to help you get started with web scraping

Image for post
Image for post
Pandas read_html() for scrapping data from HTML tables (Image by Author using canva.com)

Web scraping is the process of collecting and parsing data from the web. The Python community has come up with some pretty powerful web scrapping tools. Among them, Pandas read_html() is a quick and convenient way for scraping data from HTML tables.

In this article, you’ll learn Pandas read_html() to deal with the following common problems and should help you get started with web scraping.

  1. Reading tables from a string
  2. Reading tables from a URL
  3. Reading tables from a file
  4. Parsing date columns with parse_dates
  5. Explicitly typecast with converters
  6. MultiIndex, header, and index column
  7. Matching a table with match
  8. Filtering…


Some Pandas tricks to help you get started with data analysis

Image for post
Image for post
Photo by Athul Ben on Unsplash

Suppose you encountered a situation where you need to push all rows in a DataFrame or require to use the previous row in a DataFrame. Maybe you want to calculate the difference in consecutive rows, Pandas shift() would be an ideal way to achieve these objectives.

In this article, we’ll be going through some examples of manipulating data using Pandas shift() function. We will focus on practical problems and should help you get started with data analysis.

  1. Shifting values with periods
  2. Shifting time-series data with freq
  3. A practical example: calculating the difference in consecutive rows
  4. A practical example: calculating the…

B. Chen

Machine Learning practitioner | Formerly health informatics at University of Oxford | Ph.D.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store