Introduction In this tutorial, we will show you how to remove the leading and trailing whitespaces from a string column of a PySpark DataFrame. In order to do this, we will use the functions trim(), ltrim() and rtrim() of PySpark. Import Libraries First, we import the following python modules: from...
Introduction In this tutorial, we will show you how to group and concatenate strings in a PySpark DataFrame. In order to do this, we will use the groupBy() method in combination with the functions concat_ws(), collect_list() and array_distinct() of PySpark. Import Libraries First, we import the following...
Introduction FastAPI, a high-performance Python web framework, coupled with Docker, a powerful containerization tool, can significantly boost the efficiency of your development workflow. In this blog post, we'll walk you through the process of setting up a FastAPI project using a Dockerfile, providing a flexible and scalable solution...
Introduction One of the key tasks in data analysis is grouping data to gain insights and make informed decisions. In this tutorial, we will show you how to group the rows of a Pandas DataFrame and apply different aggregations on the grouped data. In order to do this, we will...
Introduction One of the key tasks in data analysis is grouping data to gain insights and make informed decisions. In this tutorial, we will show you how to group the rows of a PySpark DataFrame and apply different aggregations on the grouped data. In order to do this, we will...
Introduction Data comes in different forms, each with its own characteristics and challenges. Basically, there are three main categories of data: Structured, Semi-structured and Unstructured Data. In this tutorial, we explore the characteristics and some examples for each kind of data. Structured Data First, let's have a look...
Introduction FastAPI has quickly gained popularity as a modern, fast and easy-to-use Python web framework for building RESTful APIs. In this tutorial, we show you step-by-step how to set up a FastAPI project. Prerequisites First of all, make sure you have Python installed on your system. Furthermore, it is recommended...
Introduction In this tutorial, we show you step-by-step how to set up a Python virtual environment on your Mac. Why Use a Virtual Environment? A virtual environment allows you to create isolated environments for different Python projects. This helps to avoid conflicts between project dependencies and ensures that each project...
Introduction In this tutorial, we want to explain the characteristics of a Data Lakehouse. In order to do this, we will take a closer look at the key features of Data Lakes and Data Warehouses and how a Data Lakehouse combines the best of both worlds. Definition At its core,...