Real Python

Data Version Control With Python and DVC

2020-08-19T14:00:00+00:00

Machine learning and data science come with a set of problems that are different from what you’ll find in traditional software engineering. Version control systems help developers manage changes to source code. But data version control, managing changes to models and datasets, isn’t so well established.

It’s not easy to keep track of all the data you use for experiments and the models you produce. Accurately reproducing experiments that you or others have done is a challenge. Many teams are actively developing tools and frameworks to solve these problems.

In this tutorial, you’ll learn how to:

Use a tool called DVC to tackle some of these challenges
Track and version your datasets and models
Share a single development computer between teammates
Create reproducible machine learning experiments

This tutorial includes several examples of data version control techniques in action. To follow along, you can get the repository with the sample code by clicking the following link:

Get the Source Code: Click here to get the source code you'll use to learn about data version control with DVC in this tutorial.

What Is Data Version Control?

In standard software engineering, many people need to work on a shared codebase and handle multiple versions of the same code. This can quickly lead to confusion and costly mistakes.

To address this problem, developers use version control systems, such as Git, that help keep team members organized.

In a version control system, there’s a central repository of code that represents the current, official state of the project. A developer can make a copy of that project, make some changes, and request that their new version become the official one. Their code is then reviewed and tested before it’s deployed to production.

These quick feedback cycles can happen many times per day in traditional development projects. But similar conventions and standards are largely missing from commercial data science and machine learning. Data version control is a set of tools and processes that tries to adapt the version control process to the data world.

Having systems in place that allow people to work quickly and pick up where others have left off would increase the speed and quality of delivered results. It would enable people to manage data transparently, run experiments effectively, and collaborate with others.

Note: An experiment in this context means either training a model or running operations on a dataset to learn something from it.

One tool that helps researchers govern their data and models and run reproducible experiments is DVC, which stands for Data Version Control.

What Is DVC?

DVC is a command-line tool written in Python. It mimics Git commands and workflows to ensure that users can quickly incorporate it into their regular Git practice. If you haven’t worked with Git before, then be sure to check out Introduction to Git and GitHub for Python Developers. If you’re familiar with Git but would like to take your skills to the next level, then check out Advanced Git Tips for Python Developers.

DVC is meant to be run alongside Git. In fact, the git and dvc commands will often be used in tandem, one after the other. While Git is used to store and version code, DVC does the same for data and model files.

Git can store code locally and also on a hosting service like GitHub, Bitbucket, or GitLab. Likewise, DVC uses a remote repository to store all your data and models. This is the single source of truth, and it can be shared amongst the whole team. You can get a local copy of the remote repository, modify the files, then upload your changes to share with team members.

The remote repository can be on the same computer you’re working on, or it can be in the cloud. DVC supports most major cloud providers, including AWS, GCP, and Azure. But you can set up a DVC remote repository on any server and connect it to your laptop. There are safeguards to keep members from corrupting or deleting the remote data.

When you store your data and models in the remote repository, a .dvc file is created. A .dvc file is a small text file that points to your actual data files in remote storage.

The .dvc file is lightweight and meant to be stored with your code in GitHub. When you download a Git repository, you also get the .dvc files. You can then use those files to get the data associated with that repository. Large data and model files go in your DVC remote storage, and small .dvc files that point to your data go in GitHub.

The best way to understand DVC is to use it, so let’s dive in. You’ll explore the most important features by working through several examples. Before you start, you’ll need to set up an environment to work in and then get some data.

Set Up Your Working Environment

In this tutorial, you’ll learn how to use DVC by practicing on examples that work with image data. You’ll play around with lots of image files and train a machine learning model that recognizes what an image contains.

To work through the examples, you’ll need to have Python and Git installed on your system. You can follow the Python 3 Installation and Setup Guide to install Python on your system. To install Git, you can read through Installing Git.

Read the full article at https://realpython.com/python-data-version-control/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Real Python Office Hours

2020-08-18T14:00:00+00:00

The Real Python Office Hours is a weekly hangout where members of Real Python get the chance to meet fellow Pythonistas to chat about your learning progress, ask questions, and discuss Python tips & tricks via screen sharing.

In this video course, you can:

View previous office hours meetings
Download the chat transcript from each meeting
Explore resources and bonus materials discussed during each meeting

Office Hours Schedule & Registration: Join us live for the next members-only Q&A session with the Real Python Team: View upcoming Office Hours events »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

A Practical Introduction to Web Scraping in Python

2020-08-17T16:11:05+00:00

Web scraping is the process of collecting and parsing raw data from the Web, and the Python community has come up with some pretty powerful web scraping tools.

The Internet hosts perhaps the greatest source of information—and misinformation—on the planet. Many disciplines, such as data science, business intelligence, and investigative reporting, can benefit enormously from collecting and analyzing data from websites.

In this tutorial, you’ll learn how to:

Parse website data using string methods and regular expressions
Parse website data using an HTML parser
Interact with forms and other website components

Note: This tutorial is adapted from the chapter “Interacting With the Web” in Python Basics: A Practical Introduction to Python 3.

The book uses Python’s built-in IDLE editor to create and edit Python files and interact with the Python shell, so you will see occasional references to IDLE throughout this tutorial. However, you should have no problems running the example code from the editor and environment of your choice.

Free Bonus: Click here to get our free Python Cheat Sheet that shows you the basics of Python 3, like working with data types, dictionaries, lists, and Python functions.

Scrape and Parse Text From Websites

Collecting data from websites using an automated process is known as web scraping. Some websites explicitly forbid users from scraping their data with automated tools like the ones you’ll create in this tutorial. Websites do this for two possible reasons:

The site has a good reason to protect its data. For instance, Google Maps doesn’t let you request too many results too quickly.
Making many repeated requests to a website’s server may use up bandwidth, slowing down the website for other users and potentially overloading the server such that the website stops responding entirely.

Important: Before using your Python skills for web scraping, you should always check your target website’s acceptable use policy to see if accessing the website with automated tools is a violation of its terms of use. Legally, web scraping against the wishes of a website is very much a gray area.

Please be aware that the following techniques may be illegal when used on websites that prohibit web scraping.

Let’s start by grabbing all the HTML code from a single web page. You’ll use a page on Real Python that’s been set up for use with this tutorial.

Your First Web Scraper

One useful package for web scraping that you can find in Python’s standard library is urllib, which contains tools for working with URLs. In particular, the urllib.request module contains a function called urlopen() that can be used to open a URL within a program.

In IDLE’s interactive window, type the following to import urlopen():

>>>

>>> from urllib.request import urlopen

The web page that we’ll open is at the following URL:

>>>

>>> url = "http://olympus.realpython.org/profiles/aphrodite"

To open the web page, pass url to urlopen():

>>>

>>> page = urlopen(url)

urlopen() returns an HTTPResponse object:

>>>

>>> page
<http.client.HTTPResponse object at 0x105fef820>

To extract the HTML from the page, first use the HTTPResponse object’s .read() method, which returns a sequence of bytes. Then use .decode() to decode the bytes to a string using UTF-8:

>>>

>>> html_bytes = page.read()
>>> html = html_bytes.decode("utf-8")

Now you can print the HTML to see the contents of the web page:

>>>

>>> print(html)
<html>
<head>
<title>Profile: Aphrodite</title>
</head>
<body bgcolor="yellow">
<center>
<br><br>
<img src="/static/aphrodite.gif" />
<h2>Name: Aphrodite</h2>
<br><br>
Favorite animal: Dove
<br><br>
Favorite color: Red
<br><br>
Hometown: Mount Olympus
</center>
</body>
</html>

Read the full article at https://realpython.com/python-web-scraping-practical-introduction/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #22: Create Cross-Platform Python GUI Apps With BeeWare

2020-08-14T12:00:00+00:00

Do you want to distribute your Python applications to other users who don't have or even use Python? Maybe you're interested in seeing your Python application run on iOS or Android mobile devices. This week on the show we have Russell Keith-Magee, the founder and maintainer of the BeeWare project. Russell talks about Briefcase, a tool that converts a Python application into native installers on macOS, Windows, Linux, and mobile devices.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python Community Interview With Bruno Oliveira

2020-08-12T14:00:00+00:00

Today I’m joined by Bruno Oliveira, who is perhaps most well known for being a pytest core developer. In this interview, we cover migrating a large codebase from C++ to Python, how to get started with pytest, and his love of Dark Souls.

Ricky: Welcome to Real Python, Bruno. I’m glad you could join us. Let’s start in the same manner we do with all our guests: How’d you get into programming, and when did you start using Python?

Bruno: Hi, Ricky. Thanks for having me.

I started programming twenty-three years ago or so. I was just getting into computers when a friend of mine showed me this book about Visual Basic, and I was absolutely amazed that you could write your own calculator.

Sometime later, my dad bought me the Delphi 3 Bible. I devoured that, and after a while I started to program in DirectX and make very simple games.

After that, I went into college, and in the second semester I managed to get an internship to work on a Delphi application for image processing. After the internship, I joined ESSS, which is the company I work at to this day.

My role at ESSS as a technical leader is to manage the technical aspects of the projects I work on, including design and day-to-day code reviews. I’m involved in four projects currently.

Along with the other technical leaders, I also develop and oversee our high-level efforts, such as migrating all the code from one Python version to the next, solving problems in our CI, implementing development workflows, and many others.

At ESSS, we develop engineering applications specifically for the oil and gas industry. When I joined, everything was coded in C++, and the team developed everything themselves, including a multiplatform GUI library. A year or so after I was hired, we started looking into Python (version 2.4 at the time) and how we could use it together with PyQt for quickly developing our applications.

Read the full article at https://realpython.com/interview-bruno-oliveira/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Identify Invalid Python Syntax

2020-08-11T14:00:00+00:00

Python is known for its simple syntax. However, when you’re learning Python for the first time or when you’ve come to Python with a solid background in another programming language, you may run into some things that Python doesn’t allow. If you’ve ever received a SyntaxError when trying to run your Python code, then this guide can help you. Throughout this course, you’ll see common examples of invalid syntax in Python and learn how to resolve the issue.

By the end of this course, you’ll be able to:

Identify invalid syntax in Python
Make sense of SyntaxError tracebacks
Resolve invalid syntax or prevent it altogether

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Pass by Reference in Python: Background and Best Practices

2020-08-10T14:00:00+00:00

After gaining some familiarity with Python, you may notice cases in which your functions don’t modify arguments in place as you might expect, especially if you’re familiar with other programming languages. Some languages handle function arguments as references to existing variables, which is known as pass by reference. Other languages handle them as independent values, an approach known as pass by value.

If you’re an intermediate Python programmer who wishes to understand Python’s peculiar way of handling function arguments, then this tutorial is for you. You’ll implement real use cases of pass-by-reference constructs in Python and learn several best practices to avoid pitfalls with your function arguments.

In this tutorial, you’ll learn:

What it means to pass by reference and why you’d want to do so
How passing by reference differs from both passing by value and Python’s unique approach
How function arguments behave in Python
How you can use certain mutable types to pass by reference in Python
What the best practices are for replicating pass by reference in Python

Free Bonus: 5 Thoughts On Python Mastery, a free course for Python developers that shows you the roadmap and the mindset you'll need to take your Python skills to the next level.

Defining Pass by Reference

Before you dive into the technical details of passing by reference, it’s helpful to take a closer look at the term itself by breaking it down into components:

Pass means to provide an argument to a function.
By reference means that the argument you’re passing to the function is a reference to a variable that already exists in memory rather than an independent copy of that variable.

Since you’re giving the function a reference to an existing variable, all operations performed on this reference will directly affect the variable to which it refers. Let’s look at some examples of how this works in practice.

Below, you’ll see how to pass variables by reference in C#. Note the use of the ref keyword in the highlighted lines:

using System;

// Source:
// https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/passing-parameters
class Program
{
    static void Main(string[] args)
    {
        int arg;

        // Passing by reference.
        // The value of arg in Main is changed.
        arg = 4;
        squareRef(ref arg);
        Console.WriteLine(arg);
        // Output: 16
    }

    static void squareRef(ref int refParameter)
    {
        refParameter *= refParameter;
    }
}

As you can see, the refParameter of squareRef() must be declared with the ref keyword, and you must also use the keyword when calling the function. Then the argument will be passed in by reference and can be modified in place.

Python has no ref keyword or anything equivalent to it. If you attempt to replicate the above example as closely as possible in Python, then you’ll see different results:

>>>

>>> def main():
...     arg = 4
...     square(arg)
...     print(arg)
...
>>> def square(n):
...     n *= n
...
>>> main()
4

In this case, the arg variable is not altered in place. It seems that Python treats your supplied argument as a standalone value rather than a reference to an existing variable. Does this mean Python passes arguments by value rather than by reference?

Not quite. Python passes arguments neither by reference nor by value, but by assignment. Below, you’ll quickly explore the details of passing by value and passing by reference before looking more closely at Python’s approach. After that, you’ll walk through some best practices for achieving the equivalent of passing by reference in Python.

Contrasting Pass by Reference and Pass by Value

When you pass function arguments by reference, those arguments are only references to existing values. In contrast, when you pass arguments by value, those arguments become independent copies of the original values.

Let’s revisit the C# example, this time without using the ref keyword. This will cause the program to use the default behavior of passing by value:

using System;

// Source:
// https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/passing-parameters
class Program
{
    static void Main(string[] args)
    {
        int arg;

        // Passing by value.
        // The value of arg in Main is not changed.
        arg = 4;
        squareVal(arg);
        Console.WriteLine(arg);
        // Output: 4
    }

    static void squareVal(int valParameter)
    {
        valParameter *= valParameter;
    }
}

Here, you can see that squareVal() doesn’t modify the original variable. Rather, valParameter is an independent copy of the original variable arg. While that matches the behavior you would see in Python, remember that Python doesn’t exactly pass by value. Let’s prove it.

Python’s built-in id() returns an integer representing the memory address of the desired object. Using id(), you can verify the following assertions:

Function arguments initially refer to the same address as their original variables.
Reassigning the argument within the function gives it a new address while the original variable remains unmodified.

In the below example, note that the address of x initially matches that of n but changes after reassignment, while the address of n never changes:

>>>

>>> def main():
...     n = 9001
...     print(f"Initial address of n: {id(n)}")
...     increment(n)
...     print(f"  Final address of n: {id(n)}")
...
>>> def increment(x):
...     print(f"Initial address of x: {id(x)}")
...     x += 1
...     print(f"  Final address of x: {id(x)}")
...
>>> main()
Initial address of n: 140562586057840
Initial address of x: 140562586057840
  Final address of x: 140562586057968
  Final address of n: 140562586057840

Read the full article at https://realpython.com/python-pass-by-reference/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #21: Exploring K-means Clustering and Building a Gradebook With Pandas

2020-08-07T12:00:00+00:00

Do you want to learn the how and when of implementing K-means clustering in Python? Would you like to practice your pandas skills with a real-world project? This week on the show, David Amos is back with another batch of PyCoder’s Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

What Are Python Wheels and Why Should You Care?

2020-08-05T14:00:00+00:00

Python .whl files, or wheels, are a little-discussed part of Python, but they’ve been a boon to the installation process for Python packages. If you’ve installed a Python package using pip, then chances are that a wheel has made the installation faster and more efficient.

Wheels are a component of the Python ecosystem that helps to make package installs just work. They allow for faster installations and more stability in the package distribution process. In this tutorial, you’ll dive into what wheels are, what good they serve, and how they’ve gained traction and made Python even more of a joy to work with.

In this tutorial, you’ll learn:

What wheels are and how they compare to source distributions
How you can use wheels to control the package installation process
How to create and distribute wheels for your own Python packages

You’ll see examples using popular open source Python packages from both the user’s and the developer’s perspective.

Free Bonus: Click here to get a Python Cheat Sheet and learn the basics of Python 3, like working with data types, dictionaries, lists, and Python functions.

Setup

To follow along, activate a virtual environment and make sure you have the latest versions of pip, wheel, and setuptools installed:

$ python -m venv env && source ./env/bin/activate
$ python -m pip install -U pip wheel setuptools
Successfully installed pip 20.1 setuptools-46.1.3 wheel-0.34.2

That’s all you need to experiment with installing and building wheels!

Python Packaging Made Better: An Intro to Python Wheels

Before you learn how to package a project into a wheel, it helps to know what using one looks like from the user’s side. It may sound backward, but a good way to learn how wheels work is to start by installing something that isn’t a wheel.

You can start this experiment by installing a Python package into your environment just as you might normally do. In this case, install uWSGI version 2.0.x:

 1 $ python -m pip install 'uwsgi==2.0.*'
 2 Collecting uwsgi==2.0.*
 3   Downloading uwsgi-2.0.18.tar.gz (801 kB)
 4      |████████████████████████████████| 801 kB 1.1 MB/s
 5 Building wheels for collected packages: uwsgi
 6   Building wheel for uwsgi (setup.py) ... done
 7   Created wheel for uwsgi ... uWSGI-2.0.18-cp38-cp38-macosx_10_15_x86_64.whl
 8   Stored in directory: /private/var/folders/jc/8_hqsz0x1tdbp05 ...
 9 Successfully built uwsgi
10 Installing collected packages: uwsgi
11 Successfully installed uwsgi-2.0.18

To fully install uWSGI, pip progresses through several distinct steps:

On line 3, it downloads a TAR file (tarball) named uwsgi-2.0.18.tar.gz that’s been compressed with gzip.
On line 6, it takes the tarball and builds a .whl file through a call to setup.py.
On line 7, it labels the wheel uWSGI-2.0.18-cp38-cp38-macosx_10_15_x86_64.whl.
On line 10, it installs the actual package after having built the wheel.

The tar.gz tarball that pip retrieves is a source distribution, or sdist, rather than a wheel. In some ways, a sdist is the opposite of a wheel.

Note: If you see an error with the uWSGI installation, you may need to install the Python development headers.

A source distribution contains source code. That includes not only Python code but also the source code of any extension modules (usually in C or C++) bundled with the package. With source distributions, extension modules are compiled on the user’s side rather than the developer’s.

Source distributions also contain a bundle of metadata sitting in a directory called <package-name>.egg-info. This metadata helps with building and installing the package, but user’s don’t really need to do anything with it.

From the developer’s perspective, a source distribution is what gets created when you run the following command:

$ python setup.py sdist

Now try installing a different package, chardet:

 1 $ python -m pip install 'chardet==3.*'
 2 Collecting chardet
 3   Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
 4      |████████████████████████████████| 133 kB 1.5 MB/s
 5 Installing collected packages: chardet
 6 Successfully installed chardet-3.0.4

You can see a noticeably different output than the uWSGI install.

Read the full article at https://realpython.com/python-wheels/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Practical Recipes for Working With Files in Python

2020-08-04T14:00:00+00:00

Python has several built-in modules and functions for handling files. These functions are spread out over several modules such as os, os.path, shutil, and pathlib, to name a few. This course gathers in one place many of the functions you need to know in order to perform the most common operations on files in Python.

In this course, you’ll learn how to:

Retrieve file properties
Create directories
Match patterns in filenames
Traverse directory trees
Make temporary files and directories
Delete files and directories
Copy, move, or rename files and directories
Create and extract ZIP archives

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Customize the Django Admin With Python

2020-08-03T14:00:00+00:00

The Django framework comes with a powerful administrative tool called admin. You can use it out of the box to quickly add, delete, or edit any database model from a web interface. But with a little extra code, you can customize the Django admin to take your admin capabilities to the next level.

In this tutorial, you’ll learn how to:

Add attribute columns in the model object list
Link between model objects
Add filters to the model object list
Make model object lists searchable
Modify the object edit forms
Override Django admin templates

Free Bonus: Click here to get access to a free Django Learning Resources Guide (PDF) that shows you tips and tricks as well as common pitfalls to avoid when building Python + Django web applications.

Prerequisites

To get the most out of this tutorial, you’ll need some familiarity with Django, particularly model objects. As Django isn’t part of the standard Python library, it’s best if you also have some knowledge of pip and pyenv (or an equivalent virtual environment tool). To learn more about these topics, check out the following resources:

You may also be interested in one of the many available Django tutorials.

The code snippets in this tutorial were tested against Django 3.0.7. All the concepts predate Django 2.0, so they should work in whatever version you’re using, but minor differences may exist.

Setting Up the Django Admin

The Django admin provides a web-based interface for creating and managing database model objects. To see it in action, you’ll first need a Django project and some object models. Install Django inside a clean virtual environment:

$ python -m pip install django
$ django-admin startproject School
$ cd School
$ ./manage.py startapp core
$ ./manage.py migrate
$ ./manage.py createsuperuser
Username: admin
Email address: admin@example.com
Password:
Password (again):

You first create a new Django project named School with an app called core. Then you migrate the authentication tables and create an administrator. Access to the Django admin screens is restricted to users with staff or superuser flags, so you use the createsuperuser management command to create a superuser.

You also need to modify School/settings.py to include the new app named core:

# School/settings.py
# ...

INSTALLED_APPS = [
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
    "core",    # Add this line
]

The core app directory will start with the following files inside:

core/
│
├── migrations/
│   └── __init__.py
│
├── __init__.py
├── admin.py
├── apps.py
├── models.py
├── tests.py
└── views.py

You’re interested in two of these files:

models.py defines your database models.
admin.py registers your models with the Django admin.

To demonstrate the outcome when you customize the Django admin, you’ll need some models. Edit core/models.py:

from django.core.validators import MinValueValidator, MaxValueValidator
from django.db import models

class Person(models.Model):
    last_name = models.TextField()
    first_name = models.TextField()
    courses = models.ManyToManyField("Course", blank=True)

    class Meta:
        verbose_name_plural = "People"

class Course(models.Model):
    name = models.TextField()
    year = models.IntegerField()

    class Meta:
        unique_together = ("name", "year", )

class Grade(models.Model):
    person = models.ForeignKey(Person, on_delete=models.CASCADE)
    grade = models.PositiveSmallIntegerField(
        validators=[MinValueValidator(0), MaxValueValidator(100)])
    course = models.ForeignKey(Course, on_delete=models.CASCADE)

These models represent students taking courses at a school. A Course has a name and a year in which it was offered. A Person has a first and last name and can take zero or more courses. A Grade contains a percentage score that a Person received on a Course.

Here’s a model diagram showing the relationships between the objects:

The underlying table names in the database are slightly different from this, but they’re related to the models shown above.

Read the full article at https://realpython.com/customize-django-admin-python/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #20: Building PDFs in Python with ReportLab

2020-07-31T12:00:00+00:00

Have you wanted to generate advanced reports as PDFs using Python? Maybe you want to build documents with tables, images, or fillable forms. This week on the show we have Mike Driscoll to talk about his book "ReportLab - PDF Processing with Python."

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Namespaces and Scope in Python

2020-07-29T14:00:00+00:00

This tutorial covers Python namespaces, the structures used to organize the symbolic names assigned to objects in a Python program.

The previous tutorials in this series have emphasized the importance of objects in Python. Objects are everywhere! Virtually everything that your Python program creates or acts on is an object.

An assignment statement creates a symbolic name that you can use to reference an object. The statement x = 'foo' creates a symbolic name x that refers to the string object 'foo'.

In a program of any complexity, you’ll create hundreds or thousands of such names, each pointing to a specific object. How does Python keep track of all these names so that they don’t interfere with one another?

In this tutorial, you’ll learn:

How Python organizes symbolic names and objects in namespaces
When Python creates a new namespace
How namespaces are implemented
How variable scope determines symbolic name visibility

Free Bonus: 5 Thoughts On Python Mastery, a free course for Python developers that shows you the roadmap and the mindset you'll need to take your Python skills to the next level.

Namespaces in Python

A namespace is a collection of currently defined symbolic names along with information about the object that each name references. You can think of a namespace as a dictionary in which the keys are the object names and the values are the objects themselves. Each key-value pair maps a name to its corresponding object.

Namespaces are one honking great idea—let’s do more of those!

— The Zen of Python, by Tim Peters

As Tim Peters suggests, namespaces aren’t just great. They’re honking great, and Python uses them extensively. In a Python program, there are four types of namespaces:

Built-In
Global
Enclosing
Local

These have differing lifetimes. As Python executes a program, it creates namespaces as necessary and deletes them when they’re no longer needed. Typically, many namespaces will exist at any given time.

The Built-In Namespace

The built-in namespace contains the names of all of Python’s built-in objects. These are available at all times when Python is running. You can list the objects in the built-in namespace with the following command:

>>>

>>> dir(__builtins__)
['ArithmeticError', 'AssertionError', 'AttributeError',
 'BaseException','BlockingIOError', 'BrokenPipeError', 'BufferError',
 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError',
 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError',
 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError',
 'Exception', 'False', 'FileExistsError', 'FileNotFoundError',
 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError',
 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError',
 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt',
 'LookupError', 'MemoryError', 'ModuleNotFoundError', 'NameError', 'None',
 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError',
 'OverflowError', 'PendingDeprecationWarning', 'PermissionError',
 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning',
 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration',
 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError',
 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError',
 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError',
 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError',
 'Warning', 'ZeroDivisionError', '_', '__build_class__', '__debug__',
 '__doc__', '__import__', '__loader__', '__name__', '__package__',
 '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray',
 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex',
 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate',
 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset',
 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input',
 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list',
 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct',
 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr',
 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod',
 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']

You’ll see some objects here that you may recognize from previous tutorials—for example, the StopIteration exception, built-in functions like max() and len(), and object types like int and str.

The Python interpreter creates the built-in namespace when it starts up. This namespace remains in existence until the interpreter terminates.

The Global Namespace

The global namespace contains any names defined at the level of the main program. Python creates the global namespace when the main program body starts, and it remains in existence until the interpreter terminates.

Strictly speaking, this may not be the only global namespace that exists. The interpreter also creates a global namespace for any module that your program loads with the import statement. For further reading on main functions and modules in Python, see these resources:

You’ll explore modules in more detail in a future tutorial in this series. For the moment, when you see the term global namespace, think of the one belonging to the main program.

The Local and Enclosing Namespaces

As you learned in the previous tutorial on functions, the interpreter creates a new namespace whenever a function executes. That namespace is local to the function and remains in existence until the function terminates.

Read the full article at https://realpython.com/python-namespaces-scope/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python's None: Null in Python

2020-07-28T14:00:00+00:00

If you have experience with other programming languages, like C or Java, then you’ve probably heard of the concept of null. Many languages use this to represent a pointer that doesn’t point to anything, to denote when a variable is empty, or to mark default parameters that you haven’t yet supplied. null is often defined to be 0 in those languages, but null in Python is different.

Python uses the keyword None to define null objects and variables. While None does serve some of the same purposes as null in other languages, it’s another beast entirely. As the null in Python, None is not defined to be 0 or any other value. In Python, None is an object and a first-class citizen!

In this course, you’ll learn:

What None is and how to test for it
When and why to use None as a default parameter
What None and NoneType mean in your traceback
How to use None in type checking
How null in Python works under the hood

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Generative Adversarial Networks: Build Your First Models

2020-07-27T14:00:00+00:00

Generative adversarial networks (GANs) are neural networks that generate material, such as images, music, speech, or text, that is similar to what humans produce.

GANs have been an active topic of research in recent years. Facebook’s AI research director Yann LeCun called adversarial training “the most interesting idea in the last 10 years” in the field of machine learning. Below, you’ll learn how GANs work before implementing two generative models of your own.

In this tutorial, you’ll learn:

What a generative model is and how it differs from a discriminative model
How GANs are structured and trained
How to build your own GAN using PyTorch
How to train your GAN for practical applications using a GPU and PyTorch

Let’s get started!

Free Bonus: 5 Thoughts On Python Mastery, a free course for Python developers that shows you the roadmap and the mindset you'll need to take your Python skills to the next level.

What Are Generative Adversarial Networks?

Generative adversarial networks are machine learning systems that can learn to mimic a given distribution of data. They were first proposed in a 2014 NeurIPS paper by deep learning expert Ian Goodfellow and his colleagues.

GANs consist of two neural networks, one trained to generate data and the other trained to distinguish fake data from real data (hence the “adversarial” nature of the model). Although the idea of a structure to generate data isn’t new, when it comes to image and video generation, GANs have provided impressive results such as:

Style transfer using CycleGAN, which can perform a number of convincing style transformations on images
Generation of human faces with StyleGAN, as demonstrated on the website This Person Does Not Exist

Structures that generate data, including GANs, are considered generative models in contrast to the more widely studied discriminative models. Before diving into GANs, you’ll look at the differences between these two kinds of models.

Discriminative vs Generative Models

If you’ve studied neural networks, then most of the applications you’ve come across were likely implemented using discriminative models. Generative adversarial networks, on the other hand, are part of a different class of models known as generative models.

Discriminative models are those used for most supervised classification or regression problems. As an example of a classification problem, suppose you’d like to train a model to classify images of handwritten digits from 0 to 9. For that, you could use a labeled dataset containing images of handwritten digits and their associated labels indicating which digit each image represents.

During the training process, you’d use an algorithm to adjust the model’s parameters. The goal would be to minimize a loss function so that the model learns the probability distribution of the output given the input. After the training phase, you could use the model to classify a new handwritten digit image by estimating the most probable digit the input corresponds to, as illustrated in the figure below:

You can picture discriminative models for classification problems as blocks that use the training data to learn the boundaries between classes. They then use these boundaries to discriminate an input and predict its class. In mathematical terms, discriminative models learn the conditional probability P(y|x) of the output y given the input x.

Besides neural networks, other structures can be used as discriminative models such as logistic regression models and support vector machines (SVMs).

Generative models like GANs, however, are trained to describe how a dataset is generated in terms of a probabilistic model. By sampling from a generative model, you’re able to generate new data. While discriminative models are used for supervised learning, generative models are often used with unlabeled datasets and can be seen as a form of unsupervised learning.

Using the dataset of handwritten digits, you could train a generative model to generate new digits. During the training phase, you’d use some algorithm to adjust the model’s parameters to minimize a loss function and learn the probability distribution of the training set. Then, with the model trained, you could generate new samples, as illustrated in the following figure:

To output new samples, generative models usually consider a stochastic, or random, element that influences the samples generated by the model. The random samples used to drive the generator are obtained from a latent space in which the vectors represent a kind of compressed form of the generated samples.

Unlike discriminative models, generative models learn the probability P(x) of the input data x, and by having the distribution of the input data, they’re able to generate new data instances.

Note: Generative models can also be used with labeled datasets. When they are, they’re trained to learn the probability P(x|y) of the input x given the output y. They can also be used for classification tasks, but in general, discriminative models perform better when it comes to classification.

You can find more information on the relative strengths and weaknesses of discriminative and generative classifiers in the article On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes.

Although GANs have received a lot of attention in recent years, they’re not the only architecture that can be used as a generative model. Besides GANs, there are various other generative model architectures such as:

Read the full article at https://realpython.com/generative-adversarial-networks/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #19: Advanced Python Import Techniques and Managing Users in Django

2020-07-24T12:00:00+00:00

Would you like to clearly understand what's happening when you use the Python import keyword? Do you want to use modules more effectively to structure your code? Or maybe you're ready to move to the next level with your Django project by adding user management. This week on the show, David Amos is back with another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python Packages: Five Real Python Favorites

2020-07-22T14:00:00+00:00

Python has a vast ecosystem of packages, modules, and libraries that you can use to create your application. Some of these packages and modules are included with your Python installation and are collectively known as the standard library.

The standard library consists of modules that provide standardized solutions to common programming problems. They’re great building blocks for applications across many disciplines. However, many developers prefer to use alternative packages, or extensions, that may improve on the usability and usefulness of what’s in the standard library.

In this tutorial, you’ll meet some of the authors at Real Python and learn about packages they like to use in place of more common packages in the standard library.

The packages you’ll learn about in this tutorial are:

pudb: An advanced, text-based visual debugger
requests: A beautiful API for making HTTP requests
parse: An intuitive, readable text matcher
dateutil: An extension for the popular datetime library
typer: An intuitive command-line interface parser

You’ll begin by looking at a visual and powerful alternative to pdb.

Free Download: Get a sample chapter from Python Tricks: The Book that shows you Python's best practices with simple examples you can apply instantly to write more beautiful + Pythonic code.

`pudb` for Visual Debugging

Christopher Trudeau is an author and course creator at Real Python. At work he’s a consultant who helps organizations improve their technical teams. At home, he spends his time with board games and photography.

I spend a lot of time working SSHed into remote machines, so I can’t take advantage of most IDEs. My debugger of choice is pudb, which has a text-based user interface. I find its interface intuitive and easy to use.

Python ships with pdb, which was inspired by gdb, which itself was inspired by dbx. While pdb does the job, the strongest thing it has going for it is that it ships with Python. Because it’s based on the command line, you have to remember a lot of shortcut keys and can only see small amounts of the source code at a time.

An alternative Python package for debugging is pudb. It displays a full screen of source code along with useful debugging information. It has the added benefit of making me feel nostalgic for the old days when I coded in Turbo Pascal:

The interface is divided into two main parts. The left panel is for source code, and the right panel is for context information. The right-hand side is subdivided into three sections:

Variables
Stack
Breakpoints

Everything you need in a debugger is available on one screen.

Interacting With `pudb`

You can install pudb through pip:

$ python -m pip install pudb

If you’re using Python 3.7 or above, then you can take advantage of breakpoint() by setting the PYTHONBREAKPOINT environment variable to pudb.set_trace. If you’re using a Unix-based operating system such as Linux or macOS, then you set the variable as follows:

$ export PYTHONBREAKPOINT=pudb.set_trace

If you’re based in Windows, the command is different:

C:\> set PYTHONBREAKPOINT=pudb.set_trace

Alternatively, you can insert import pudb; pudb.set_trace() directly into your code.

Read the full article at https://realpython.com/python-packages/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Mastering Python's Built-in time Module

2020-07-21T14:00:00+00:00

The Python time module provides many ways of representing time in code, such as objects, numbers, and strings. It also provides functionality other than representing time, like waiting during code execution and measuring the efficiency of your code.

This course will walk you through the most commonly used functions and objects in time.

By the end of this course, you’ll be able to:

Understand core concepts at the heart of working with dates and times, such as epochs, time zones, and daylight savings time
Represent time in code using floats, tuples, and struct_time
Convert between different time representations
Suspend thread execution
Measure code performance using perf_counter()

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

K-Means Clustering in Python: A Practical Guide

2020-07-20T14:00:00+00:00

The k-means clustering method is an unsupervised machine learning technique used to identify clusters of data objects in a dataset. There are many different types of clustering methods, but k-means is one of the oldest and most approachable. These traits make implementing k-means clustering in Python reasonably straightforward, even for novice programmers and data scientists.

If you’re interested in learning how and when to implement k-means clustering in Python, then this is the right place. You’ll walk through an end-to-end example of k-means clustering using Python, from preprocessing the data to evaluating results.

In this tutorial, you’ll learn:

What k-means clustering is
When to use k-means clustering to analyze your data
How to implement k-means clustering in Python with scikit-learn
How to select a meaningful number of clusters

Click the link below to download the code you’ll use to follow along with the examples in this tutorial and implement your own k-means clustering pipeline:

Download the sample code: Click here to get the code you'll use to learn how to write a k-means clustering pipeline in this tutorial.

What Is Clustering?

Clustering is a set of techniques used to partition data into groups, or clusters. Clusters are loosely defined as groups of data objects that are more similar to other objects in their cluster than they are to data objects in other clusters. In practice, clustering helps identify two qualities of data:

Meaningfulness
Usefulness

Meaningful clusters expand domain knowledge. For example, in the medical field, researchers applied clustering to gene expression experiments. The clustering results identified groups of patients who respond differently to medical treatments.

Useful clusters, on the other hand, serve as an intermediate step in a data pipeline. For example, businesses use clustering for customer segmentation. The clustering results segment customers into groups with similar purchase histories, which businesses can then use to create targeted advertising campaigns.

Note: You’ll learn about unsupervised machine learning techniques in this tutorial. If you’re interested in learning more about supervised machine learning techniques, then check out Logistic Regression in Python.

There are many other applications of clustering, such as document clustering and social network analysis. These applications are relevant in nearly every industry, making clustering a valuable skill for professionals working with data in any field.

Overview of Clustering Techniques

You can perform clustering using many different approaches—so many, in fact, that there are entire categories of clustering algorithms. Each of these categories has its own unique strengths and weaknesses. This means that certain clustering algorithms will result in more natural cluster assignments depending on the input data.

Note: If you’re interested in learning about clustering algorithms not mentioned in this section, then check out A Comprehensive Survey of Clustering Algorithms for an excellent review of popular techniques.

Selecting an appropriate clustering algorithm for your dataset is often difficult due to the number of choices available. Some important factors that affect this decision include the characteristics of the clusters, the features of the dataset, the number of outliers, and the number of data objects.

You’ll explore how these factors help determine which approach is most appropriate by looking at three popular categories of clustering algorithms:

Partitional clustering
Hierarchical clustering
Density-based clustering

It’s worth reviewing these categories at a high level before jumping right into k-means. You’ll learn the strengths and weaknesses of each category to provide context for how k-means fits into the landscape of clustering algorithms.

Partitional Clustering

Partitional clustering divides data objects into nonoverlapping groups. In other words, no object can be a member of more than one cluster, and every cluster must have at least one object.

These techniques require the user to specify the number of clusters, indicated by the variable k. Many partitional clustering algorithms work through an iterative process to assign subsets of data points into k clusters. Two examples of partitional clustering algorithms are k-means and k-medoids.

These algorithms are both nondeterministic, meaning they could produce different results from two separate runs even if the runs were based on the same input.

Partitional clustering methods have several strengths:

Read the full article at https://realpython.com/k-means-clustering-python/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #18: Ten Years of Flask: Conversation With Creator Armin Ronacher

2020-07-17T12:00:00+00:00

This week on the show we have Armin Ronacher to talk about the first 10 years of Flask. Armin talks about the origins of Flask and the components that make up the framework. He talks about what goes into documenting a framework or API. He also talks about the community working on the ongoing development of Flask.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Grow Your Python Portfolio With 13 Intermediate Project Ideas

2020-07-14T14:00:00+00:00

Now that you know the basics of Python, you can put that knowledge to use by building projects to put in your portfolio. The trick is finding project ideas that are just right for your level. Creating a variety of applications is a way to demonstrate your knowledge and share it with others.

In this course, you’ll get 13 project ideas that you can work on as an intermediate Python developer. The project ideas cover a wide range of application type and can be built using a command line, web based, or graphical user interface. You’ll see examples of existing applications for inspiration, and get links to Python libraries and resources to assist in your application development.

In this course, you’ll learn:

The importance of building projects
The major platforms you can build projects for
Thirteen project ideas you can work on
Some tips for working on projects

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #17: Linear Programming, PySimpleGUI, and More

2020-07-10T12:00:00+00:00

Are you familiar with linear programming, and how it can be used to solve resource optimization problems? Would you like to free your Python code from a clunky command line and start making convenient graphical interfaces for your users? This week on the show, David Amos is back with another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Pointers and Objects in Python

2020-07-07T14:00:00+00:00

If you’ve ever worked with lower-level languages like C or C++, then you may have heard of pointers. Pointers are essentially variables that hold the memory address of another variable. They allow you to create great efficiency in parts of your code but can lead to various memory management bugs.

You’ll learn about Python’s object model and see why pointers in Python don’t really exist. For the cases where you need to mimic pointer behavior, you’ll learn ways to simulate pointers in Python without managing memory.

In this course, you’ll:

Learn why pointers in Python don’t exist
Explore the difference between C variables and Python names
Simulate pointers in Python
Experiment with real pointers using ctypes

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #16: Thinking in Pandas: Python Data Analysis the Right Way

2020-07-03T12:00:00+00:00

Are you using the Python library Pandas the right way? Do you wonder about getting better performance, or how to optimize your data for analysis? What does normalization mean? This week on the show we have Hannah Stepanek to discuss her new book "Thinking in Pandas".

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Unicode in Python: Working With Character Encodings

2020-06-30T14:00:00+00:00

Python’s Unicode support is strong and robust, but it takes some time to master. There are many ways of encoding text into binary data, and in this course you’ll learn a bit of the history of encodings. You’ll also spend time learning the intricacies of Unicode, UTF-8, and how to use them when programming Python. You’ll practice with multiple examples and see how smooth working with text and binary data in Python can be!

By the end of this course, you’ll know:

What an encoding is
What ASCII is
How binary displays as octal and hex values
How UTF-8 encodes a code point
How to combine code points into a single glyph
Which built-in functions can help you

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #15: Python Regular Expressions, Views vs Copies in Pandas, and More

2020-06-26T12:00:00+00:00

Have you wanted to learn Regular Expressions in Python, but don't know where to start? Have you stumbled into the dreaded pink SettingWithCopyWarning in Pandas? This week on the show, we have David Amos from the Real Python team to discuss a recent two-part series on Regex in Python. We also talk about another recent article on the site about views vs copies in Pandas. David also brings a few other articles and projects from the wider Python community for us to discuss.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Creating a Discord Bot in Python

2020-06-23T14:00:00+00:00

In a world where video games are so important to so many people, communication and community around games are vital. Discord offers both of those and more in one well-designed package. In this course, you’ll learn how to make a Discord bot in Python so that you can make the most of this fantastic platform.

By the end of this course you’ll learn:

What Discord is and why it’s so valuable
How to make a Discord bot through the Developer Portal
How to create Discord connections
How to handle events
How to accept commands and validate assumptions
How to interact with various Discord APIs

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #14: Going Serverless with Python

2020-06-19T12:00:00+00:00

Would you like to run your Python code in the cloud without having to become an infrastructure engineer? Do you want to have Python functions that run when triggered by specific events? This week on the show we have Anthony Chu to discuss serverless computing and running python functions in the cloud. Anthony Chu is program manager for Microsoft's Azure Functions.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python Generators 101

2020-06-16T14:00:00+00:00

Have you ever had to work with a dataset so large that it overwhelmed your machine’s memory? Or maybe you have a complex function that needs to maintain an internal state every time it’s called, but the function is too small to justify creating its own class. In these cases and more, generators and the Python yield statement are here to help.

By the end of this course, you’ll know:

What generators are and how to use them
How to create generator functions and expressions
How the Python yield statement works
How to use multiple Python yield statements in a generator function
How to use advanced generator methods
How to build data pipelines with multiple generators

If you’re a beginner or intermediate Pythonista and you’re interested in learning how to work with large datasets in a more Pythonic way, then this is the course for you.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #13: PDFs in Python and Projects on the Raspberry Pi

2020-06-12T12:00:00+00:00

Have you wanted to work with PDF files in Python? Maybe you want to extract text, merge and concatenate files, or even create PDFs from scratch. Are you interested in building hardware projects using a Raspberry Pi? This week on the show we have David Amos from the Real Python team to discuss his recent article on working with PDFs. David also brings a few other articles from the wider Python community for us to discuss.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Real Python

Data Version Control With Python and DVC

What Is Data Version Control?

What Is DVC?

Set Up Your Working Environment

Real Python Office Hours

A Practical Introduction to Web Scraping in Python

Scrape and Parse Text From Websites

Your First Web Scraper

The Real Python Podcast – Episode #22: Create Cross-Platform Python GUI Apps With BeeWare

Python Community Interview With Bruno Oliveira

Identify Invalid Python Syntax

Pass by Reference in Python: Background and Best Practices

Defining Pass by Reference

Contrasting Pass by Reference and Pass by Value

The Real Python Podcast – Episode #21: Exploring K-means Clustering and Building a Gradebook With Pandas

What Are Python Wheels and Why Should You Care?

Setup

Python Packaging Made Better: An Intro to Python Wheels

Practical Recipes for Working With Files in Python

Customize the Django Admin With Python

Prerequisites

Setting Up the Django Admin

The Real Python Podcast – Episode #20: Building PDFs in Python with ReportLab

Namespaces and Scope in Python

Namespaces in Python

The Built-In Namespace

The Global Namespace

The Local and Enclosing Namespaces

Python's None: Null in Python

Generative Adversarial Networks: Build Your First Models

What Are Generative Adversarial Networks?

Discriminative vs Generative Models

The Real Python Podcast – Episode #19: Advanced Python Import Techniques and Managing Users in Django

Python Packages: Five Real Python Favorites

pudb for Visual Debugging

Interacting With pudb

Mastering Python's Built-in time Module

K-Means Clustering in Python: A Practical Guide

What Is Clustering?

Overview of Clustering Techniques

Partitional Clustering

The Real Python Podcast – Episode #18: Ten Years of Flask: Conversation With Creator Armin Ronacher

Grow Your Python Portfolio With 13 Intermediate Project Ideas

The Real Python Podcast – Episode #17: Linear Programming, PySimpleGUI, and More

Pointers and Objects in Python

The Real Python Podcast – Episode #16: Thinking in Pandas: Python Data Analysis the Right Way

Unicode in Python: Working With Character Encodings

The Real Python Podcast – Episode #15: Python Regular Expressions, Views vs Copies in Pandas, and More

Creating a Discord Bot in Python

The Real Python Podcast – Episode #14: Going Serverless with Python

Python Generators 101

The Real Python Podcast – Episode #13: PDFs in Python and Projects on the Raspberry Pi

`pudb` for Visual Debugging

Interacting With `pudb`