A list of premium posts for academy members containing hands-on tutorials, best practices, career advices and learning paths.
📘Introduction Welcome to this hands-on FastMCP tutorial! In this guide, we’ll walk step-by-step through how to add a tool to your FastMCP Server. By the end, you’ll have a working server with a custom tool that can be called by AI clients. ✅ Prerequisites Before you begin, make sure...
📘 Introduction Welcome to this hands-on OpenAI Agents tutorial! In this guide, we’ll walk you step-by-step through creating your first AI agent using the OpenAI Agents SDK — perfect for beginners who want to build intelligent, autonomous assistants quickly. 💡 What is the OpenAI Agents SDK? The OpenAI Agents SDK is a...
📘 Introduction Welcome to this hands-on OpenAI GPT tutorial! In this guide, we’ll walk you step-by-step through sending your first prompt to a GPT model using the OpenAI Python SDK — perfect for beginners who want to start building AI-powered applications fast. 💡 What is the OpenAI Python SDK? The OpenAI Python...
📘Introduction Welcome to this hands-on FastMCP tutorial! In this guide, we’ll walk you step-by-step through creating your first FastMCP server from scratch — perfect for beginners who want to jump into MCP quickly. ✅ Prerequisites Before you begin, make sure you have the following: 🐍✅ Installed Python 📦✅ Installed Pip 🌐✅ Created and activated...
📘 Introduction In this hands-on dbt tutorial, we’ll walk you step-by-step through generating and serving interactive documentation for your dbt project. 📌 This is a must-know topic for the dbt Analytics Engineering Certification Exam, so mastering it now puts you one step closer to passing the exam and leveling up your...
📘 Introduction Welcome to this hands-on FastMCP tutorial! In this guide, we’ll take you step-by-step through setting up and using FastMCP to accelerate your AI projects with ease and efficiency. ✅ Prerequisites Before you begin, make sure you have the following: 🐍✅ Installed Python 📦✅ Installed Pip 🌐✅ Created and activated a virtual environment...
📘 Introduction In this hands-on dbt tutorial, we'll walk you through why data contracts are critical for your dbt models — and how to add them to your dbt models step by step. 📌 This is a must-know topic for the dbt Analytics Engineering Certification Exam, so mastering it now puts...
📘 Introduction In the rapidly evolving world of artificial intelligence, interoperability is emerging as a critical challenge. With multiple agents, tools, memory systems, and models working in tandem, the need for a universal “plug-and-play” protocol is greater than ever. Enter the Model Context Protocol (MCP) — a new standard designed to unify...
📘 Introduction Data is only as useful as your ability to explore and understand it. But building an interface to analyze and visualize your DataFrame often feels like a full-blown front-end project. That’s where Streamlit saves the day. In this tutorial, you’ll learn how to build a data app...
📘 Introduction In this hands-on dbt tutorial, we'll walk you through how to add and run generic data tests in dbt using YAML configuration. These built-in tests help ensure your data quality and integrity, without writing any custom SQL. 📌 This is a must-know topic for the dbt Analytics Engineering...
📘 Introduction In this hands-on dbt tutorial, we'll walk you through how to use Common Table Expressions (CTEs) effectively in dbt models - an essential concept for modular, readable, and testable SQL. 📌 This is a must-know topic for the dbt Analytics Engineering Certification Exam, so mastering it now puts...
📘Introduction In this hands-on dbt tutorial, we'll walk you through how to describe and document your data models using schema.yml in dbt. 📌 This is a must-know topic for the dbt Analytics Engineering Certification Exam, so mastering it now puts you one step closer to passing the exam...
📘Introduction In Software Engineering, DRY stands for Don't Repeat Yourself — a fundamental principle aimed at reducing repetition, improving maintainability, and increasing code clarity. In the world of data transformation with dbt (data build tool), applying DRY principles helps ensure that your SQL code is modular, reusable, and easy...
📘 Introduction The dbt Analytics Engineering Certification Exam can be challenging—but with the right preparation strategy, you can pass with confidence. This guide brings together the best official resources and practical tutorials to help you stay focused, study smarter, and succeed. 📋1️⃣ Read the official dbt Study Guide Begin with...
📘 Introduction dbt gives you the power to transform raw data into clean, trusted models. But writing SQL is just the starting point. To truly get value from dbt, you need practices that keep your code organized, your models reliable, and your team moving fast. Whether you're a beginner...
📘 Introduction Chatbots are transforming the way we interact with data, services, and even entertainment. But building an intuitive interface for your chatbot can often feel like a front-end headache. That’s where Streamlit shines. It helps you create interactive, elegant UIs for your Python scripts—without touching HTML or CSS....
📘Introduction Once you've created models in dbt, you can reference them using the ref() function. This is fundamental to how dbt builds dependencies and orchestrates model execution. In this hands-on tutorial, we’ll show you how to use the ref()function within a sample project. 📌 This is a...
📘 Introduction Choosing the right Python package manager is a key decision for any developer working in Python. For years, pip has been the default choice—simple, reliable, and deeply integrated with the Python ecosystem. But a new contender has entered the ring: uv, a blazing-fast package manager developed by the...
📘 Introduction Creating interactive web apps with Streamlit is incredibly easy — even if you're not a front-end developer. Once you've mastered the basics, Streamlit’s widget system becomes your playground. These widgets let users interact with your data and models in real time. In this post, you&...
📘 Introduction Data scientists and analysts often struggle to share their work interactively. Whether it's model results or beautiful visualizations, the gap between code and a usable interface can be frustrating. That's where Streamlit comes in. Streamlit is a powerful Python framework that turns your data scripts...
📘Introduction In this step-by-step tutorial, we’ll walk you through how to install and use packages in dbt to supercharge your data transformations with reusable macros, models, and integrations. 📌 This is a must-know topic for the dbt Analytics Engineering Certification Exam, so mastering it now puts you one step closer...
📘 Introduction Connecting Claude Desktop with Atlassian Confluence streamlines your workflow by enabling Claude to retrieve, summarize, and publish content directly to Confluence pages. Thanks to the Model Context Protocol (MCP), this integration is seamless. In this tutorial, we’ll show you how to set it up using Atlassian’s remote...
📘Introduction Claude, the conversational AI from Anthropic, is gaining popularity. While many users access Claude through their browsers, installing the Claude desktop app on your Mac can significantly streamline your workflow. This guide walks you through exactly how to do it—step-by-step. 🤖 What is Claude? Claude is an AI assistant...
📘Introduction If you're new to dbt (data build tool) and want to transform raw data into clean, analytics-ready tables using Python, you're in the right place. In this step-by-step tutorial, we’ll walk you through how to build your first Python model in dbt, step by...
📘Introduction If you're new to dbt (data build tool) and want to transform raw data into clean, analytics-ready tables — you're in the right place. In this step-by-step tutorial, we’ll walk you through how to build your first SQL model in dbt, step by step. 📌 This...
📘 Introduction Prompt engineering has become a foundational skill in the age of generative AI. As large language models (LLMs) like GPT-4, Claude, and others power a new wave of intelligent applications, the way we communicate with these models matters more than ever. But what exactly is prompt engineering? Why is...
📘 Introduction Artificial Intelligence (AI) is no longer science fiction—it's part of our everyday lives. From personalized content to smart assistants, AI is transforming industries and society. But myths and misconceptions can make AI seem either all-powerful or frighteningly mysterious. This guide cuts through the confusion, revealing the...
📘 Introduction Crafting a great prompt is like asking the perfect question — it can make all the difference in how helpful and accurate an AI chatbot’s response is. Whether you're using ChatGPT, Claude, Gemini, or another tool, the quality of your prompt directly impacts the quality of the...
📘Introduction AI chatbots have rapidly evolved into powerful everyday assistants—making tasks easier, faster, and more enjoyable. Whether you're looking to boost productivity, spark creativity, or simply save time, tools like ChatGPT, Claude, and Gemini offer practical support you can tap into right now—no technical skills needed....
📘 Introduction Artificial Intelligence is undergoing a significant shift. For years, generative models have captured headlines — from creating text to generating images. But a new wave is emerging: Agentic AI. So, how does Agentic AI differ from Generative AI? Are they competing paradigms — or complementary forces in the AI ecosystem? In...
📘 Introduction Artificial Intelligence is evolving rapidly. From rule-based systems to generative models, we are now entering a new frontier: Agentic AI. But what exactly is Agentic AI? How does it differ from traditional AI systems, and what opportunities or risks does it bring? In this post, we'll explore...
📘Introduction Large language models (LLMs) have revolutionized artificial intelligence, unlocking new levels of understanding and creativity. But AI’s journey is far from overnight — it’s been shaped by decades of innovation, setbacks, and breakthroughs. In this post, we’ll explore the pivotal moments in AI’s history — from the...
📘Introduction FastAPI is a modern, high-performance web framework for building APIs with Python. In real-world applications, you'll often need to connect to a relational database like PostgreSQL to store and retrieve data. But building that connection securely and cleanly—especially across development, staging, and production environments—requires a...
Introduction In the age of data-driven decision making, having a powerful data architecture is crucial. The Medallion Architecture is a proven data architecture pattern that helps in organizing data across different layers of refinement. When combined with dbt (data build tool), it becomes a powerful and scalable way to manage...
Introduction AI is incredible. It can help you write emails, explain complex ideas, brainstorm, and even code. But sometimes, it gets things completely wrong — and says it like it’s 100% right. This strange behavior is known as an AI hallucination: when an AI gives an answer that sounds correct...
Introduction Retrieval-Augmented Generation (RAG) is an advanced technique in natural language processing that enhances language models by enabling them to retrieve external information in real time. This leads to more accurate and context-aware responses, making RAG ideal for tasks that require up-to-date or domain-specific knowledge, such as customer support, enterprise...
Introduction Keeping the data fresh in your data warehouse is crucial. In dbt (data build tool), source freshness checks help ensure that the data coming from your sources is up-to-date and trustworthy, enabling teams to confidently build reliable models downstream. This is where defining source freshness in the sources.yml...
Introduction Once you've defined sources in dbt, you can reference them using the source() function. This is essential for dbt to identify dependencies between sources and models. In this hands-on tutorial, we’ll show you how to use the source() function with a sample project. 📌 This is a...
Introduction In dbt (data build tool), sources represent raw data tables that already exist in your data warehouse - often provided by upstream systems like operational databases, APIs or other applications. Defining these sources in sources.yml is a crucial step in your data project. This tutorial will guide you...
Introduction FastAPI is a modern, fast (high-performance) web framework for building APIs with Python. When building real-world applications, you’ll often need to manage sensitive information like API keys, database URLs, or feature toggles that should vary across environments (development, testing, production). Using environment variables and Pydantic’s BaseSettings is...
Introduction Seeds in dbt are a simple and powerful way to load static data - such as reference tables, lookup data, or mapping tables - directly into your data warehouse. In this tutorial, you'll learn what seeds are and how to use them effectively. Since this topic is...
Introduction The Hugging Face Inference API makes it easy to send prompts to large language models (LLMs) hosted on the Hugging Face Hub. By combining this with Gradio, you can quickly build interactive chatbots and demos with a simple, web-based user interface—without worrying about backend frameworks. In this tutorial,...
Introduction The Hugging Face Inference API makes it easy to send prompts to large language models (LLMs) hosted on the Hugging Face Hub. By combining this with FastAPI—a modern Python web framework—you can build scalable, production-ready APIs that serve LLM-powered responses to your applications. In this tutorial, you’...
Introduction Want to start with dbt core but don’t know where to begin? Don’t worry! In this tutorial, we'll walk through setting up a new dbt project from scratch - we cover the entire process from creating a virtual environment to initializing your project and verifying...
Introduction The Hugging Face Inference API is a powerful service that lets you interact with large language models (LLMs) hosted on the Hugging Face Hub. Whether you’re building chatbots, summarization tools, or other AI-powered applications, the Inference API makes it simple to send prompts to an LLM and receive...
Introduction shadcn/ui is a modern component library built for Next.js and React applications. It offers beautifully designed, accessible UI components that you can customize and extend to fit your design system. Whether you're prototyping a startup app or building an enterprise-grade product, shadcn/ui helps you...
Introduction Modern data work doesn’t always require complex infrastructure. If you're looking for a fast, local-first analytics stack, dbt + DuckDB is a powerful combination. In this tutorial, we'll walk through connecting dbt to DuckDB — perfect for analysts, developers, and data enthusiasts who want simplicity without...
Introduction In dbt, the dbt_project.yml file is the backbone of your project configuration. It controls how dbt behaves, where it looks for different resource types, and how your models are materialized and organized. In this tutorial, we'll walk through a sample dbt_project.yml file and...
Introduction Next.js is a powerful React framework that enables developers to build fast, scalable web applications with server-side rendering, static site generation, and API routes—all out of the box. Whether you're building a personal blog or an enterprise-grade application, Next.js offers the flexibility and performance...
Introduction When working with PySpark DataFrames, understanding the statistical properties of your data is crucial for data exploration and preprocessing. PySpark provides the describe() and summary() functions to generate useful summary statistics. In this tutorial, we’ll explore how to use both functions to get insights into our dataset. 📥 Import...
Introduction Microsoft Fabric enables seamless data integration by allowing users to create connections to various data sources. For the DP-600 certification exam, understanding how to set up a data connection in Fabric is essential for managing data efficiently. In this tutorial, you'll learn how to create a new...
Introduction dlt (data load tool) is a powerful Python package that simplifies data ingestion and helps you build efficient data pipelines. In the Extract, Load, Transform (ELT) process, dlt is particularly suited for the Extract (E) and Load (L) stages. In this tutorial, we'll guide you through the...
Introduction In dbt, one of the most essential commands is dbt build. In this tutorial, we’ll dive into the dbt build command, exploring its syntax, functionality, and practical usage. Since this topic is relevant for the dbt Analytics Engineering Certification Exam, this guide will be a valuable resource on...
Introduction Microsoft Fabric provides a powerful platform for managing and analyzing data efficiently. One essential capability within a Fabric warehouse is the ability to create tables, which form the foundation for structuring and storing data. Understanding how to define and create tables is crucial for managing datasets, running analytical queries,...
Introduction Microsoft Fabric provides a robust platform for data management and analytics. One fundamental aspect of managing a Fabric warehouse is the ability to create schemas. Schemas help organize tables and other database objects within a warehouse, improving structure, security, and manageability. For those preparing for the DP-600 certification exam,...
Introduction Microsoft Fabric provides a robust environment for managing and transforming data within a data warehouse. One of its powerful features is the ability to create stored procedures, which allow for encapsulating SQL logic that can be reused across multiple operations. Stored procedures simplify complex queries, automate repetitive tasks, and...
Introduction Microsoft Fabric provides a robust environment for managing and transforming data within a data warehouse. One of its powerful features is the ability to create functions in a warehouse, enabling reusable logic that simplifies complex queries and enhances efficiency. For the DP-600 certification exam, understanding how to create and...
Introduction An essential feature of Microsoft Fabric is the ability to create views in a data warehouse, enabling data transformation without physical duplication. Views simplify and aggregate data, improving accessibility and efficiency for analysis. For the DP-600 certification exam, knowing how to create and manage views is crucial. This task...
Introduction Transforming existing columns and creating new ones are essential skills in Power BI. In this hands-on tutorial, we’ll explore how to efficiently add custom columns in both Power Query Editor and the Power BI interface. Since this topic is relevant for the PL-300 exam, this guide will be...
Introduction Using Quick Measures in Power BI allows you to perform powerful calculations effortlessly. In this hands-on Power BI tutorial, you'll learn how to create quick measures without manually writing DAX. Since this topic is relevant for the PL-300 exam, this guide will be a valuable resource for...
Introduction When working with PySpark DataFrames, handling different data types correctly is essential for data preprocessing. Mismatched or incorrect data types can lead to errors in Spark operations such as filtering, aggregations, and machine learning workflows. In this tutorial, we’ll explore how to convert column data types in a...
Introduction Microsoft Fabric offers multiple storage and processing solutions for different analytical needs, including lakehouses, warehouses, and eventhouses. For the DP-600 certification exam, understanding when to use each option is crucial for designing efficient data architectures. In this tutorial, you'll learn how to differentiate between lakehouses, warehouses, and...
Introduction Statistical functions are fundamental for analyzing and interpreting data in Power BI. In this hands-on Power BI tutorial, we'll explore how to use essential statistical functions in DAX that are commonly used for data analysis. Since this topic is relevant for the PL-300 exam, this guide will...
Introduction Selecting the right column data types in Power BI is essential for accurate data analysis and visualization. In this hands-on Power BI tutorial, we'll explore key data types, how to set them using the Power Query Editor and Power BI Desktop. Since this topic is relevant for...
Introduction When working with data in Power BI, it’s essential to assess its quality, distribution, and structure before performing transformations. Power Query provides built-in tools to help analyze data statistics and column properties, ensuring data accuracy and consistency. In this hands-on Power BI tutorial, we’ll explore three key...
Introduction Errors are an inevitable part of working with data. When preparing and transforming data, you may encounter various errors that can disrupt your workflow. Understanding how to identify, troubleshoot, and resolve these errors is crucial. In this hands-on Power BI tutorial, we’ll explore different types of errors and...
Introduction Microsoft Fabric provides powerful deployment capabilities for managing changes and automating releases across development, test, and production environments. For the DP-600 certification exam, understanding how to configure deployment pipelines in a Fabric workspace is essential to enable smooth transitions between development stages and ensure consistent data and reporting experiences....
Introduction When working with Power BI, one of the key decisions you need to make is how to connect to your data. The two primary storage modes - DirectQuery and Import - each have their strengths and limitations. Choosing the right mode can significantly impact your report’s performance, flexibility,...
Introduction Slicers in Power BI are essential for filtering and analyzing data, but managing multiple slicers can be a hassle. Instead of clearing them one by one, you can create a Clear Slicer button that allows users to reset all slicers with a single click. This improves usability, enhances the...
Introduction Microsoft Fabric provides powerful version control capabilities for managing changes and tracking history in a workspace. For the DP-600 certification exam, understanding how to configure version control in a Fabric workspace is essential to enable collaborative development and maintain code integrity. In this tutorial, you'll learn how...
Introduction Effective titles are key to engaging reports, providing context and clarity to your audience. Enhance your Power BI reports with dynamic visual titles that adjust based on filter and slicer selections. This guide also serves as a valuable resource for those preparing for the Microsoft PL-300 exam, as dynamic...
Introduction If you're working with Power BI, you know that an optimized data model is crucial for performance and efficiency. That’s where Tabular Editor comes in—an essential tool for streamlining your data modeling process. In this hands-on tutorial, we’ll guide you through the installation and...
Introduction In this hands-on Power BI tutorial, we will dive deep into the REMOVEFILTERS function, including its syntax, use cases, and a practical example. Whether you're enhancing your DAX knowledge or preparing for the Microsoft PL-300 (Power BI Data Analyst) exam, mastering the REMOVEFILTERS function empowers you to...
Introduction When working with data in Python, two of the most popular tools available are Pandas and PySpark. Both offer powerful data manipulation capabilities but are designed for different use cases. Pandas is well-suited for small to moderately large datasets, offering an intuitive API for data analysis. PySpark, on the...
Introduction Understanding the ALL function in DAX is crucial for anyone working with Power BI. In this hands-on tutorial, we will explore the ALL function in depth, including its syntax, use cases, and a practical example. Whether you're upskilling your DAX knowledge or preparing for the Microsoft PL-300...
Introduction Microsoft Fabric provides robust security controls for managing data access. For the DP-600 certification exam, mastering file-level access control is essential to restrict access to specific folders within a Fabric Lakehouse. In this tutorial, you'll learn how to implement file-level access control using OneLake data access roles...
Introduction A traffic light visual is a powerful way to represent performance measures clearly and effectively🚦. While Power BI doesn’t include a built-in traffic light visual, you can easily create one with some creativity! Whether you are a beginner or an advanced user preparing for Microsoft Exam PL-300 (Power...
Introduction Microsoft Fabric provides robust security controls for managing data access. For the DP-600 certification exam, mastering Column-Level Security (CLS) is essential to restrict access to specific columns within a table in a Microsoft Fabric Warehouse. In this tutorial, you'll learn how to implement column-level access control to...
Introduction Microsoft Fabric provides robust security controls for managing data access. For the DP-600 certification exam, mastering Object-Level Security (OLS) is essential to restrict access to specific objects in a Microsoft Fabric Warehouse, such as schemas, tables, and views. In this tutorial, you'll learn how to implement object-level...
Introduction Understanding the LOOKUPVALUE function in DAX is essential for anyone working with Power BI. In this tutorial, we will explore the LOOKUPVALUE function in depth, including its syntax, use cases, and a practical example. Whether you are a beginner looking to enhance your DAX skills or an advanced user...
Introduction Microsoft Fabric provides powerful security features to control data access at a granular level. In the DP-600 certification exam, mastering how to implement row-level access control is essential for ensuring users see only the data relevant to them while maintaining governance and compliance. In this tutorial, you'll...
Introduction A heatmap is a powerful visualization type that allows you to represent data density, trends, and patterns using color gradients. In this tutorial, we will explore how to create a heatmap without using custom visuals. You will learn how to turn a matrix into a stunning heatmap. Whether you...
Introduction Microsoft Fabric provides powerful security features to control data access at a granular level. In the DP-600 certification exam, mastering how to implement row-level access control is essential for ensuring users see only the data relevant to them while maintaining governance and compliance. In this tutorial, you'll...
Introduction Microsoft Fabric offers powerful access control mechanisms to manage item permissions, ensuring secure access and effective governance. In the DP-600 certification exam, understanding how to implement item-level access controls is essential. In this tutorial, you'll learn how to share individual items within a Fabric workspace. Item-level Access...
Introduction When working with PySpark DataFrames, handling missing or empty values is a common task in data preprocessing. In many cases, empty strings ("") should be treated as null values for better compatibility with Spark operations, such as filtering, aggregations, and machine learning workflows. In this tutorial, we’ll...
Introduction In dbt, one of the most essential commands is dbt test. In this tutorial, we’ll dive into the dbt test command, exploring its syntax, functionality, and practical usage. This tutorial is also a valuable resource for those studying for the dbt Analytics Engineering Certification Exam. 💡Explanation of dbt...
Introduction Microsoft Fabric provides robust access control mechanisms to manage data security and user permissions effectively. In the DP-600 certification exam, understanding how to implement workspace-level access controls is crucial for securing sensitive data and ensuring appropriate governance. In this tutorial, you'll learn how to manage workspace access,...
Introduction When working with data in PySpark, you might often encounter scenarios where a single column contains multiple pieces of information, such as a combination of names, categories, or attributes. In such cases, it is essential to split these values into separate columns for better data organization and analysis. In...
Introduction Understanding the RELATED function in DAX is crucial for anyone working with Power BI. In this tutorial, we will explore the RELATED function in depth, including its syntax, use cases and a practical example. Whether you are a beginner looking to improve your DAX skills or an advanced user...
Introduction Data Transformation is a crucial step in any data project, ensuring that raw data is turned into accurate, consistent, and usable data. One tool that has become especially valuable in this process is dbt (Data Build Tool). In this tutorial, we’ll explore what dbt is, how it simplifies...
Introduction Parsing JSON strings with PySpark is an essential task when working with large datasets in JSON format. By transforming JSON data into a structured format, you can enable efficient processing and analysis. PySpark provides a powerful way to parse these JSON strings and extract their contents into separate columns,...
Introduction In dbt, one of the most essential commands is dbt run. In this tutorial, we’ll dive deep into the dbt run command, exploring its syntax, functionality, and practical usage. This tutorial is also a valuable resource for those preparing for the dbt Analytics Engineering Certification Exam. 💡Explanation of...
Introduction In data processing, it's common to find timestamp fields as strings. Converting these string representations into proper timestamp formats is crucial for accurate data analysis and processing. In this tutorial, we will explore how to convert a string to a timestamp column in a PySpark DataFrame. Import...
Introduction In this tutorial, we’ll dive into the concept of Materializations in dbt and explore the different types available. Understanding materializations is key to optimizing how data is stored, queried, and updated within your dbt projects. Whether you're just starting with dbt or preparing for the dbt...
Introduction In data processing, it's common to find date fields as strings. Converting these string representations into proper date formats is crucial for accurate data analysis and processing. In this tutorial, we will explore how to convert a string to a date column in a PySpark DataFrame. Import...
Introduction dbt comes in two versions - dbt Core and dbt Cloud. While both provide the core functionality for data transformation, they serve different purposes and are suited to different requirements. In this tutorial, we’ll dive into the features of dbt Core and dbt Cloud, highlighting the key differences...
Introduction Microsoft Azure provides a powerful platform for deploying and managing web applications at scale. Whether you're building a simple website or a complex web service, Azure makes it easy to set up, deploy, and maintain your web app. In this tutorial, we’ll walk through creating a...
Introduction When dealing with large datasets in PySpark, it's common to encounter situations where you need to manipulate string data within your DataFrame columns. One such common operation is extracting a portion of a string—also known as a substring—from a column. In this tutorial, we will...
Introduction Creating an App Service Plan in Microsoft Azure is essential for hosting web apps, mobile backends, and APIs. The App Service Plan defines the region, features, and pricing tier your app will use. By understanding how to set up and configure the App Service Plan, you can ensure your...
Introduction Azure Data Lake Storage (ADLS) is a scalable and fully managed cloud storage service designed for big data and analytics workloads. One of the core components of ADLS is the Blob container, which serves as a logical unit for organizing and storing data. In this tutorial, we’ll walk...
Introduction Azure Data Lake Storage Gen2 (ADLS Gen2) is an essential part of the Azure ecosystem for managing big data workloads. It combines the best features of Azure Blob Storage and Azure Data Lake Storage, making it an ideal solution for enterprise-level data storage, processing, and analytics. In this tutorial,...
Introduction Azure Data Lake Storage (ADLS) is a powerful cloud-based data storage solution designed to handle large-scale analytics, accommodating both structured and unstructured data. Its flexible architecture and integration with Azure services make it an ideal platform for organizations looking to leverage data for advanced analytics and business intelligence. In...
Introduction FastAPI is a modern web framework that empowers developers to build web APIs quickly and efficiently using Python. A POST endpoint in FastAPI allows clients to submit data to the server, often for creating new resources. One part of a POST request ist the Request Body, which contains the...
Introduction Creating a resource group in Microsoft Azure is one of the first steps when organizing your cloud resources. Resource groups help you manage and control access to various Azure services like virtual machines, databases, and networking components. In this guide, we'll walk through the process of creating...
Introduction FastAPI is a modern web framework that empowers developers to build web APIs quickly and efficiently using Python. A GET endpoint allows clients to retrieve data from a server. In a GET request, query parameters allow passing additional information to an endpoint via the URL's query string,...
Introduction GraphQL is a powerful query language for APIs that offers greater flexibility and efficiency compared to REST. While REST relies on multiple endpoints for different resources, GraphQL uses a single endpoint to fetch and manipulate data, allowing clients to request exactly what they need. In Microsoft Fabric, you can...
Introduction FastAPI is a modern web framework that empowers developers to build web APIs quickly and efficiently using Python. A GET endpoint allows clients to retrieve data from a server. In a GET request, path parameters can be used to pass values directly in the URL to retrieve specific resources....
Introduction In data processing and analysis with PySpark, it's often important to know the structure of your data, such as the number of rows and columns in a DataFrame. This is crucial for various operations, including data validation, transformations, and general exploration. In this tutorial, we'll...
Introduction Building an interactive chatbot interface is easier than ever with tools like Gradio. Gradio is an open-source Python library that allows you to create user-friendly interfaces for machine learning models and other Python functions with minimal effort. In this post, we'll walk through the steps of creating...
Introduction The Requests module offers a simple and user-friendly way of making HTTP requests in Python. Whether you are retrieving data from a web page, interacting with an API, or sending information to a server, the Requests module makes the task straightforward. In this tutorial, we will explore how to...
Introduction Building and deploying machine learning apps can sometimes be a complex task, but using Gradio, a Python library for quickly building interactive web apps for your ML models, simplifies it significantly. To make your app easily deployable across different environments, containerizing it using Docker is a perfect solution. In...
Introduction In this tutorial, we want to count the distinct values of a PySpark DataFrame column. In order to do this, we use the distinct().count() method and the countDistinct() function of PySpark. Import Libraries First, we import the following python modules: from pyspark.sql import SparkSession from pyspark.sql....
Introduction Gradio is an open-source Python library that allows you to create user-friendly web interfaces for machine learning models with minimal effort. It’s widely used for building demos, sharing models, and prototyping machine learning applications. In this tutorial, we'll walk you through how to set up a...
Introduction When working with complex data, you often come across nested JSON structures that are stored in a single column. These nested structures can be difficult to analyze in their raw form, so flattening the JSON column becomes essential for data preparation. Dataflows in Microsoft Fabric provide a powerful way...
Introduction The Data Science experience in Microsoft Fabric simplifies the end-to-end machine learning process by enabling users to effortlessly create, train, and deploy Machine Learning Models. In this tutorial, we will explain step-by-step how to create a Machine Learning Model in Fabric. Step 1: Sign in to Fabric First, sign...
Introduction In distributed computing environments like Apache Spark, efficient data handling is critical for performance. One useful feature for optimizing computations is broadcast variables. Broadcast variables allow you to share large read-only data across all nodes in a Spark cluster without duplicating the data for each task. In this tutorial,...
Introduction Microsoft Fabric offers a gallery of sample datasets that can be used to perform and practice Real-Time Analytics. These sample data sets offer a great opportunity to gain experience and familiarize yourself with the technologies and services in the Real-Time Intelligence Experience. In this tutorial we will show you...
Introduction In Microsoft Fabric, a workspace is essential for the creation and collection of components. To enhance the organization of these components, folders can be created within a workspace. In this tutorial we will demonstrate how to create a folder in a workspace. Goal A folder should be created in...
Introduction Installing Python is a straightforward process that gives you access to one of the most powerful programming languages available today. Whether you're running macOS, Windows or Linux, Python can be easily installed through official channels so you can start programming quickly. This tutorial will show you how...
Introduction Microsoft Fabric offers a gallery of sample datasets that can be used to perform and practice Real-Time Analytics. These sample data sets offer a great opportunity to gain experience and familiarize yourself with the technologies and services in the Real-Time Intelligence Experience. In this tutorial we will introduce the...
Introduction In this tutorial, we will explore how to read a Delta table into a PySpark DataFrame. Goal A Delta table stored in a lakehouse should be read into a PySpark DataFrame. Prequestisies ☑️ Notebook created We have already created the notebook "dlnerds_notebook". If you want to know...
Introduction In this tutorial, we will walk through the steps required to write a PySpark DataFrame to a Delta table in Microsoft Fabric. We'll start by creating an example DataFrame, then demonstrate how to write it to a Delta table. Goal An example PySpark DataFrame should be created...
Introduction In Microsoft Fabric, notebooks can interact very closely with lakehouses. In this tutorial, we will explain step-by-step how to connect a lakehouse and a notebook in Microsoft Fabric. Goal A lakehouse and a notebook should be connected. Prequestisies ☑️ Notebook created We have already created the notebook "dlnerds_notebook&...
Introduction One key component of Microsoft Fabric is the Lakehouse. In a lakehouse, custom schemas can be created to group tables together. In this tutorial, we will explain step-by-step how to create a lakehouse schema. Goal A new lakehouse schema should be created. Step 1: Sign in to Fabric First,...
Introduction Kusto Query Language (KQL) databases are essential components for real-time analytics in Microsoft Fabric. In this tutorial, we will explain step-by-step how to add a new KQL Database to an existing Eventhouse. Goal A new KQL Database should be added to an existing Eventhouse. Step 1: Open Real-Time Intelligence...
Introduction Power BI comes with the powerful formula language Data Analysis Expression (DAX) which allows the implementation of custom calculations. There are numerous operators and functions available in DAX. One important DAX function is the CONCATENATE function. With the CONCATENATE function, two text values can be merged into one text...
Introduction Microsoft Fabric is a powerful All-in-One Data Platform (SaaS) in the Azure Cloud that combines various Azure components to cover the fields of Data Integration, Data Engineering, Data Science and Business Intelligence. One key component of the Microsoft Fabric architecture is the Eventhouse. In this tutorial, we will explain...
Introduction The first step when creating a Power BI report is to connect with data sources. Power BI can connect to a wide range of data sources. This capability allows users to access and analyze data from various sources within Power BI. One important format to often deal with is...
Introduction Microsoft Fabric offers with the Data Explorer a powerful tool to visualize, analyze and explore query results. It offers an intuitive interface to create visualizations and filters. In this tutorial, we will explain step-by-step how to visualize the results of a T-SQL query using the Data Explorer in Microsoft...
Introduction In Power BI, the visualization and interpretation of data in a table can be improved by using conditional formatting with icons. This feature dynamically adds icons to a table based on the values it contains. This allows important information to be highlighted so that patterns, trends and outliers can...
Introduction Microsoft Fabric is a powerful All-in-One Data Platform (SaaS) in the Azure Cloud that combines various Azure components to cover the fields of Data Integration, Data Engineering, Data Science and Business Intelligence. One key component of the Microsoft Fabric architecture is the KQL Queryset. In this tutorial, we will...
Introduction In Power BI, the visualization and interpretation of data in a table can be improved by using conditional formatting with background colors. This feature dynamically sets the cell colors in a table based on the values it contains. This allows important information to be highlighted so that patterns, trends...
Introduction In this tutorial, we will show you how to get a randomly sampled subset of a PySpark DataFrame. In order to do this, we will use the sample() function of PySpark. What is the sample() Function? The sample() function in PySpark is used to create a new DataFrame by...
Introduction In Power BI, readability and visualization of tables can be enhanced using data bars, which provide a visual representation of numerical values for quick comparisons and trend identification. In this tutorial, we will show you how to use data bars in a table visualization in Power BI. Data The...
Introduction In Microsoft Fabric, you can schedule a data pipeline by defining specific times and frequencies for your data processing tasks. This automation helps ensure that your data is constantly updated and available at the right time. In this tutorial, we will explain step-by-step how to schedule a data pipeline...
Introduction In Power BI, you can enhance data visualization by sorting one column based on the values of another column. This feature allows you to customize the order of your data, providing clearer insights and more meaningful comparisons. In this tutorial, we will show you how to sort one column...
Introduction In Microsoft Fabric, you can schedule a dataflow by defining specific times and frequencies for your data processing tasks. This automation helps ensure that your data is constantly updated and available at the right time. In this tutorial, we will explain step-by-step how to schedule a dataflow in Microsoft...
Introduction Power BI comes with the powerful formula language Data Analysis Expression (DAX) which allows the implementation of custom calculations. There are numerous operators and functions available in DAX. One of the most important DAX functions is the CALCULATE function. The CALCULATE function enables dynamic aggregations based on specific criteria....
Introduction In Microsoft Fabric, there are several activities that can be added to a data pipeline. One import activity is the Notebook activity. In this tutorial, we will explain step-by-step how to add a Notebook activity to a data pipeline using the Data Factory user interface. Goal A CSV File...
Introduction In this tutorial, we will explain step-by-step how to query data from a Fabric lakehouse with T-SQL using an SQL analytics endpoint. Goal Data should be queried from a delta table in the lakehouse with T-SQL using an SQL analytics endpoint. Data We consider the delta table student that...
Introduction In Microsoft Fabric, there are several activities that can be added to a data pipeline. One import activity is the Copy activity. In this tutorial, we will explain step-by-step how to add a Copy activity to a data pipeline using the Data Factory user interface. Goal A CSV File...
Introduction In this tutorial, we will explain step-by-step how to save a T-SQL query as table in Microsoft Fabric. Goal An existing T-SQL query should be saved as table. Step 1: Open Data Warehouse Experience We have already signed in into Fabric and opened the Data Warehouse Experience. We have...
Introduction In today's data-driven world, understanding text data is crucial across various domains, from data analysis to engineering and architecture. However, dealing with text data often requires converting it into numerical representations for machine learning models to process efficiently. This is where embedding vectors come into play, offering...
Introduction In this tutorial, we will explain step-by-step how to save a T-SQL query as view in Microsoft Fabric. Goal An existing T-SQL query should be saved as view. Step 1: Open Data Warehouse Experience We have already signed in into Fabric and opened the Data Warehouse Experience. We have...
Introduction In this tutorial, we will show you how to concatenate multiple string columns of a PySpark DataFrame into a single column. In order to do this, we will use the functions concat() and concat_ws() of PySpark. Import Libraries First, we import the following python modules: from pyspark.sql...
Introduction In the realm of data visualization, bar charts stand tall as one of the most effective and widely used tools for conveying information. In Power BI, it is very easy to create vertical and horizontal bar charts. Furthermore a grouping can be added, which can be displayed in different...
Introduction In order to work with data in Microsoft Fabric, you first need to get data into it. There are several ways to do this. One way is to use a notebook. In this tutorial, we will explain step-by-step how to ingest data into a Fabric lakehouse using a notebook....
Introduction In this tutorial, we will explain step-by-step how to query data from a Fabric warehouse with T-SQL using the visual query editor. Goal Data should be queried from a table in the warehouse with T-SQL using the visual query editor. Data We consider the table student that is stored...
Introduction In this tutorial, we will show you how to remove the leading and trailing whitespaces from a string column of a PySpark DataFrame. In order to do this, we will use the functions trim(), ltrim() and rtrim() of PySpark. Import Libraries First, we import the following python modules: from...
Introduction In this tutorial, we will explain step-by-step how o read an Excel file into a PySpark DataFrame in Databricks. Configure Cluster First, install on a Databricks cluster the spark-excel library (also referred as com.crealytics.spark.excel). To do this, select your Databricks cluster in the "Compute"...
Introduction In this tutorial, we will explain step-by-step how to query data from a Fabric warehouse using T-SQL. Goal Data should be queried from a table in the warehouse with T-SQL. Data We consider the table student that is stored in the warehouse "dlnerds_warehouse". Step 1: Open...
Introduction In order to work with data in Microsoft Fabric, you first need to get data into it. There are several ways to do this. One way is to use a dataflow. In this tutorial, we will explain step-by-step how to ingest data into a Fabric warehouse using a dataflow....
Introduction In order to work with data in Microsoft Fabric, you first need to get data into it. There are several ways to do this. One way is to use a data pipeline. In this tutorial, we will explain step-by-step how to ingest data into a Fabric warehouse using a...
Introduction One fundamental part of Microsoft Fabric is transforming data. Whether filtering, joining, merging or grouping data, there are several options available in Fabric to perform these operations. In this tutorial, we will explain step-by-step how to group data and apply an aggregation function using a dataflow. Goal A delta...
Introduction In this tutorial, we will show you how to group and concatenate strings in a PySpark DataFrame. In order to do this, we will use the groupBy() method in combination with the functions concat_ws(), collect_list() and array_distinct() of PySpark. Import Libraries First, we import the following...
Introduction In order to work with data in Microsoft Fabric, you first need to get data into it. There are several ways to do this. One way is to use a dataflow. In this tutorial, we will explain step-by-step how to ingest data into a Fabric lakehouse using a dataflow....
Introduction In order to work with data in Microsoft Fabric, you first need to get data into it. There are several ways to do this. One way is to use a data pipeline. In this tutorial, we will explain step-by-step how to ingest data into a Fabric lakehouse using a...
Introduction In order to work with data in Microsoft Fabric, you first need to get data into it. There are several ways to do this. One way is to manually upload files or folders from your local drive into a Fabric lakehouse. In this tutorial, we will explain step-by-step how...
Introduction Power BI makes it possible to bring numbers and information to life, tell stories and make data-driven decisions. This is the world of Power BI! Whether you are already familiar with data or just starting out, Power BI is a powerful tool that allows you to turn raw data...
Introduction Data transformation is a fundamental task in any data analysis or processing pipeline. In the realm of big data processing, Apache Spark has emerged as a powerful framework for handling large-scale data processing tasks efficiently. When it comes to transforming data within Spark, developers often have to choose between...
Introduction Microsoft Fabric is a powerful All-in-One Data Platform (SaaS) in the Azure Cloud that combines various Azure components to cover the fields of Data Integration, Data Engineering, Data Science and Business Intelligence. One key component of the Microsoft Fabric architecture is the Data Pipeline. In this tutorial, we will...
Introduction Microsoft Fabric is a powerful All-in-One Data Platform (SaaS) in the Azure Cloud that combines various Azure components to cover the fields of Data Integration, Data Engineering, Data Science and Business Intelligence. One key component of the Microsoft Fabric architecture is the Dataflow. In this tutorial, we will explain...
Introduction Microsoft Fabric is a powerful All-in-One Data Platform (SaaS) in the Azure Cloud that combines various Azure components to cover the fields of Data Integration, Data Engineering, Data Science and Business Intelligence. One key component of the Microsoft Fabric architecture is the Notebook. In this tutorial, we will explain...
Introduction Microsoft Fabric is a powerful All-in-One Data Platform (SaaS) in the Azure Cloud that combines various Azure components to cover the fields of Data Integration, Data Engineering, Data Science and Business Intelligence. One key component of the Microsoft Fabric architecture is the Warehouse. In this tutorial, we will explain...
Introduction Microsoft Fabric is a powerful All-in-One Data Platform (SaaS) in the Azure Cloud that combines various Azure components to cover the fields of Data Integration, Data Engineering, Data Science and Business Intelligence. One key component of the Microsoft Fabric architecture is the Lakehouse. In this tutorial, we will explain...
Introduction Microsoft Fabric is a powerful All-in-One Data Platform (SaaS) in the Azure Cloud that combines various Azure components to cover the fields of Data Integration, Data Engineering, Data Science and Business Intelligence. One key component of the Microsoft Fabric architecture is the Workspace. In this tutorial, we will explain...
Introduction Microsoft Fabric is a powerful All-in-One Data Platform (SaaS) in the Azure Cloud that combines various Azure components to cover the fields of Data Integration, Data Engineering, Data Science and Business Intelligence. In order to explore and get to know Fabric, Microsoft offers a free trial. In this tutorial,...
Introduction In the realm of big data processing, PySpark has emerged as a powerful tool for handling large-scale datasets. Its distributed computing framework allows for efficient processing of massive volumes of data. However, despite its capabilities, performing certain data transformations in PySpark can sometimes be cumbersome and complex. That'...
Introduction Power BI offers with the Power Query Editor a powerful tool for cleaning and transforming data. One important part of data preparation is adding an Index Column. For organizing and structuring your data it is crucial that every row is uniquely identified by an ID. An Index Column enables...
Introduction Power BI comes with the powerful formula language Data Analysis Expression (DAX) which allows the implementation of custom calculations. There are numerous operators and functions available in DAX. One essential type of DAX functions are aggregation functions. These functions are essential for data analysis, especially when it comes to...
Introduction As projects grow in size and complexity, it becomes increasingly important to ensure that code remains understandable and easy to work with. One powerful tool for achieving this is the use of type hints. In this tutorial, we will explain why and how to use type hints in Python....
Introduction Autoencoders, a cornerstone of unsupervised learning, have emerged as a powerful tool in the domain of artificial intelligence. By compressing and then reconstructing input data, these neural networks uncover meaningful representations, paving the way for applications ranging from image denoising to anomaly detection. In this tutorial we want to...
Introduction The first step when creating a Power BI report is to connect with data sources. Power BI can connect to a wide range of data sources. This capability allows users to access and analyze data from various sources within Power BI. One important format to often deal with is...
Introduction Traditional RNNs, limited by their simplistic structure, have problems retaining information over longer time periods, leading to the infamous vanishing gradient problem. Long Short-Term Memory (LSTM) Networks have the impressive ability to capture and preserve long-term dependencies in sequential data. But how is an LSTM able to do this?...
Introduction Environment variables are used for securely storing and accessing sensitive data, facilitating seamless configuration management across different environments. In this tutorial, we will explore how to work with environment variables in Python. In order to do this, we will use the Python libraries os and python-dotenv. What is an...
Introduction Power BI offers with the Power Query Editor a powerful tool for cleaning and transforming data. One important part of data preparation is filtering your data. Filtering enables you to sort out irrelevant data and to reduce the amount of data. One important type of filtering is custom filtering....
Introduction Window functions in PySpark are a powerful feature for data manipulation and analysis. They allow you to perform complex calculations on subsets of data within a DataFrame, without the need for expensive joins or subqueries. In this tutorial, we will show you how to use window functions in PySpark....
Introduction In Machine Learning, one essential step is evaluating the performance of a model. For classification models, the Confusion Matrix serves as a fundamental instrument for evaluating the performance. The Confusion Matrix provides a visualization of the results of a model. Based on the information from the Confusion Matrix, some...
Introduction FastAPI, a high-performance Python web framework, coupled with Docker, a powerful containerization tool, can significantly boost the efficiency of your development workflow. In this blog post, we'll walk you through the process of setting up a FastAPI project using a Dockerfile, providing a flexible and scalable solution...
Introduction In Machine Learning, one essential step is evaluating the performance of a model. For classification models, the Confusion Matrix serves as a fundamental instrument for evaluating the performance. It provides a clear and visual summary of the prediction accuracy of a model by illustrating the correspondence between the predicted...
Introduction One common task when working with large datasets is the need to generate unique identifiers for each record. In this tutorial, we will explore how to easily add an ID column to a PySpark DataFrame. In order to do this, we use the monotonically_increasing_id() function of PySpark....
Introduction In the fast-paced world of software development, efficiency and consistency are key. Docker, a powerful containerization platform, has revolutionized the way we build, ship, and run applications. In this tutorial, we show you how to get started with Docker. Step 1: Install Docker (if not already installed) First of...
Introduction PostgreSQL is one of the most widely used database management systems. One of the easiest ways to use PostgreSQL on macOS is the Postgres.app. Postgres.app provides a simple interface for setting up a server and a command-line interface (psql) for interacting with databases via the terminal. In...
Introduction In 1997, Sepp Hochreiter and Jürgen Schmidhuber introduced Long Short-Term Memory (LSTM), a special type of Recurrent Neural Network (RNN). The motivation was to overcome the problems of traditional RNNs when dealing with long sequences. This is exactly what LSTM Networks (LSTMs) are supposed to do. The special architecture...
Introduction In the landscape of software development, Docker has emerged as a game changer, offering a streamlined approach to building, shipping, and running applications. In this blog post, we explain the fundamental concept of Docker focusing on Dockerfiles, Docker images, and containers. What is Docker? Docker is an open-source platform...
Introduction One of the key tasks in data analysis is grouping data to gain insights and make informed decisions. In this tutorial, we will show you how to group the rows of a PySpark DataFrame and apply different aggregations on the grouped data. In order to do this, we will...
Introduction Data comes in different forms, each with its own characteristics and challenges. Basically, there are three main categories of data: Structured, Semi-structured and Unstructured Data. In this tutorial, we explore the characteristics and some examples for each kind of data. Structured Data First, let's have a look...
Introduction FastAPI has quickly gained popularity as a modern, fast and easy-to-use Python web framework for building RESTful APIs. In this tutorial, we show you step-by-step how to set up a FastAPI project. Prerequisites First of all, make sure you have Python installed on your system. Furthermore, it is recommended...
Introduction Power BI is one of the most widely used BI tools. But using Power BI on the Mac can be a challenge. This is because Microsoft does not offer a version of Power BI Desktop for the Mac. Nevertheless, there are workarounds for using Power BI on the Mac....
Introduction In this tutorial, we show you step-by-step how to set up a Python virtual environment on your Mac. Why Use a Virtual Environment? A virtual environment allows you to create isolated environments for different Python projects. This helps to avoid conflicts between project dependencies and ensures that each project...
Introduction In this tutorial, we want to explain the characteristics of a Data Lakehouse. In order to do this, we will take a closer look at the key features of Data Lakes and Data Warehouses and how a Data Lakehouse combines the best of both worlds. Definition At its core,...
Introduction In this tutorial we will cover how to represent boolean functions with the McCulloch-Pitts Neuron. We look at the boolean functions AND, OR and NOT. McCulloch-Pitts Neuron We have already explained how the McCulloch-Pitts Neuron works in another post. If you haven't seen it yet, check it...
Introduction In this tutorial, we want to explode arrays into rows of a PySpark DataFrame. In order to do this, we use the explode() function and the explode_outer() function of PySpark. Import Libraries First, we import the following python modules: from pyspark.sql import SparkSession from pyspark.sql.functions...
Introduction In this tutorial, we want to explain why Generative AI (GenAI) is possible now. In order to do this, we describe the key factors that are responsible for the rise of Generative AI. Factors The key factors that enable Generative AI are availability of large datasets, computational power, and...
Introduction In this tutorial, we want to scale features of a Pandas DataFrame. In order to do this, we use the StandardScaler() class and the MinMaxScaler() class of Scikit-Learn. Import Libraries First, we import the following python modules: import pandas as pd from sklearn.datasets import fetch_california_housing from...
Introduction In this tutorial we will cover the very first and the simplest mathematical neuron model in history - the McCulloch-Pitts Neuron. We look at the architecture and functionality. History The McCulloch-Pitts-Neuron is the simplest form of a neuron model and was published in 1943 by Warren McCulloch and Walter...
Introduction In this tutorial, we want to one-hot encode a NumPy array that contains categorical values. In order to do this, we use the to_categorical() function of Keras. Import Libraries First, we import the following python modules: import numpy as np from keras.utils import to_categorical Define Data...
Introduction Machine Learning can be divided into two main types: Supervised Learning and Unsupervised Learning. In this tutorial, we want to take a closer look to these approaches and compare them to each other. Overview Both supervised learning and unsupervised learning have their own characteristics and are suitable for solving...
Introduction In this tutorial, we want to take a closer look to the most important Machine Learning algorithms and compare them to each other. Overview...
Introduction In contrast to a classic feedforward network, a Recurrent Neural Network (RNN) allows backward connections. An RNN is particularly suitable for processing time series data and is able to take into account the time dependency of data. This enables an RNN to have a kind of short-term memory. The...
Introduction In this tutorial, we want to import Stock Prices from Yahoo Finance into Python. In order to do this, we use ticker module of YFinance. YFinance The python library yfinance enables access to financial data from Yahoo Finance. Yahoo Finance provides various financial market data such as stock...
Introduction Data Scientists, Data Engineer and Data Analyst are some of the most familiar roles in the data world. In this tutorial, we want to take a closer look at each role . We look at their required skills and the technologies being used. Data Scientist...
Introduction In this tutorial, we want to explain how Generative Adversarial Neural Networks (GANs) work. In order to do this, we take a look at the functionality and the idea behind GANs. Principle Generative Adversarial Neural Networks (GANs) are a special type of Artificial Neural Networks. This kind of model...
Introduction Artificial Intelligence (AI) is developing at a rapid pace. Especially in the gaming industry, AI is now superior to humans. In this tutorial we will have a look at the games in which the AI is superior. Throughout history, there have been some groundbreaking achievements of AI. The successes...
Introduction In this tutorial, we want to explain Generative AI (GenAI). In order to do this, we describe both the terms Artificial Intelligence, Machine Learning, Deep Learning and Generative AI as well as the relationships between them. Overview First, we want to have a look to the relationships between Artificial...
Introduction In this tutorial, we want to create an Artificial Neural Network (ANN) that is able to recognize the species of penguins based on their body measurements. We will cover the entire process from Data Collection, Data Cleansing and Feature Engineering to Model Creation and Model Training. Use Case Based...
Introduction In this tutorial, we want to use regular expressions (regex) to filter, replace and extract strings of a PySpark DataFrame based on specific patterns. In order to do this, we use the rlike() method, the regexp_replace() function and the regexp_extract() function of PySpark. Import Libraries...
Introduction In this tutorial, we want to create a UDF and apply it to a PySpark DataFrame. In order to do this, we will show you two different ways: using the udf() function and using the @udf decorator. Import Libraries First, we import the following python modules: from pyspark.sql...
Introduction In this tutorial, we want to make aggregate operations on columns of a PySpark DataFrame. In order to do this, we use different aggregate functions of PySpark. Import Libraries First, we import the following python modules: from pyspark.sql import SparkSession from pyspark.sql.functions import * Create SparkSession Before...
Introduction In this tutorial, we want to concatenate multiple PySpark DataFrames. In order to do this, we use the the union() method of PySpark. Import Libraries First, we import the following python modules: from pyspark.sql import SparkSession Create SparkSession Before we can work with Pyspark, we need to create...
Introduction In this tutorial, we want to join PySpark DataFrames. In order to do this, we use the the join() method of PySpark. Import Libraries First, we import the following python modules: from pyspark.sql import SparkSession Create SparkSession Before we can work with Pyspark, we need to create a...