Real Python

Stochastic Gradient Descent Algorithm With Python and NumPy

2021-01-27T14:00:00+00:00

Stochastic gradient descent is an optimization algorithm often used in machine learning applications to find the model parameters that correspond to the best fit between predicted and actual outputs. It’s an inexact but powerful technique.

Stochastic gradient descent is widely used in machine learning applications. Combined with backpropagation, it’s dominant in neural network training applications.

In this tutorial, you’ll learn:

How gradient descent and stochastic gradient descent algorithms work
How to apply gradient descent and stochastic gradient descent to minimize the loss function in machine learning
What the learning rate is, why it’s important, and how it impacts results
How to write your own function for stochastic gradient descent

Free Bonus: 5 Thoughts On Python Mastery, a free course for Python developers that shows you the roadmap and the mindset you’ll need to take your Python skills to the next level.

Basic Gradient Descent Algorithm

The gradient descent algorithm is an approximate and iterative method for mathematical optimization. You can use it to approach the minimum of any differentiable function.

Note: There are many optimization methods and subfields of mathematical programming. If you want to learn how to use some of them with Python, then check out Scientific Python: Using SciPy for Optimization and Hands-On Linear Programming: Optimization With Python.

Although gradient descent sometimes gets stuck in a local minimum or a saddle point instead of finding the global minimum, it’s widely used in practice. Data science and machine learning methods often apply it internally to optimize model parameters. For example, neural networks find weights and biases with gradient descent.

Cost Function: The Goal of Optimization

The cost function, or loss function, is the function to be minimized (or maximized) by varying the decision variables. Many machine learning methods solve optimization problems under the surface. They tend to minimize the difference between actual and predicted outputs by adjusting the model parameters (like weights and biases for neural networks, decision rules for random forest or gradient boosting, and so on).

In a regression problem, you typically have the vectors of input variables 𝐱 = (𝑥₁, …, 𝑥ᵣ) and the actual outputs 𝑦. You want to find a model that maps 𝐱 to a predicted response 𝑓(𝐱) so that 𝑓(𝐱) is as close as possible to 𝑦. For example, you might want to predict an output such as a person’s salary given inputs like the person’s number of years at the company or level of education.

Your goal is to minimize the difference between the prediction 𝑓(𝐱) and the actual data 𝑦. This difference is called the residual.

In this type of problem, you want to minimize the sum of squared residuals (SSR), where SSR = Σᵢ(𝑦ᵢ − 𝑓(𝐱ᵢ))² for all observations 𝑖 = 1, …, 𝑛, where 𝑛 is the total number of observations. Alternatively, you could use the mean squared error (MSE = SSR / 𝑛) instead of SSR.

Both SSR and MSE use the square of the difference between the actual and predicted outputs. The lower the difference, the more accurate the prediction. A difference of zero indicates that the prediction is equal to the actual data.

SSR or MSE is minimized by adjusting the model parameters. For example, in linear regression, you want to find the function 𝑓(𝐱) = 𝑏₀ + 𝑏₁𝑥₁ + ⋯ + 𝑏ᵣ𝑥ᵣ, so you need to determine the weights 𝑏₀, 𝑏₁, …, 𝑏ᵣ that minimize SSR or MSE.

In a classification problem, the outputs 𝑦 are categorical, often either 0 or 1. For example, you might try to predict whether an email is spam or not. In the case of binary outputs, it’s convenient to minimize the cross-entropy function that also depends on the actual outputs 𝑦ᵢ and the corresponding predictions 𝑝(𝐱ᵢ):

In logistic regression, which is often used to solve classification problems, the functions 𝑝(𝐱) and 𝑓(𝐱) are defined as the following:

Again, you need to find the weights 𝑏₀, 𝑏₁, …, 𝑏ᵣ, but this time they should minimize the cross-entropy function.

Gradient of a Function: Calculus Refresher

In calculus, the derivative of a function shows you how much a value changes when you modify its argument (or arguments). Derivatives are important for optimization because the zero derivatives might indicate a minimum, maximum, or saddle point.

The gradient of a function 𝐶 of several independent variables 𝑣₁, …, 𝑣ᵣ is denoted with ∇𝐶(𝑣₁, …, 𝑣ᵣ) and defined as the vector function of the partial derivatives of 𝐶 with respect to each independent variable: ∇𝐶 = (∂𝐶/∂𝑣₁, …, ∂𝐶/𝑣ᵣ). The symbol ∇ is called nabla.

The nonzero value of the gradient of a function 𝐶 at a given point defines the direction and rate of the fastest increase of 𝐶. When working with gradient descent, you’re interested in the direction of the fastest decrease in the cost function. This direction is determined by the negative gradient, −∇𝐶.

Read the full article at https://realpython.com/gradient-descent-algorithm-python/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Evaluate Expressions Dynamically With Python eval()

2021-01-26T14:00:00+00:00

The built-in Python function eval() is used to evaluate Python expressions. You can pass a string containing Python, or a pre-compiled object into eval() and it will run the code and return the result.

Although Python’s eval() is an incredibly useful tool, the function has some important security implications that you should consider before using it. In this course, you’ll learn how eval() works and how to use it safely and effectively in your Python programs.

In this tutorial, you’ll learn:

How Python’s eval() works
How to use eval() to dynamically evaluate arbitrary string-based or compiled-code-based input
How eval() can make your code insecure and how to minimize the associated security risks

The code in this course was tested with Python 3.9.0, eval() has not changed much and older versions should be compatible.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

How to Use Python: Your First Steps

2021-01-25T16:27:13+00:00

Are you looking for a place to learn the basics of how to use Python from a beginner’s perspective? Do you want to get up and running with Python but don’t know where to start? If so, then this tutorial is for you. This tutorial focuses on the essentials you need to know to start programming with Python.

In this tutorial, you’ll learn:

What Python is and why you should use it
What basic Python syntax you should learn to start coding
How to handle errors in Python
How to get help quickly in Python
What code style you should apply in your code
Where to get extra functionalities without reinventing the wheel
Where to find quality Python content and grow your skills

You’ll also have the opportunity to create your first Python program and run it on your computer. Finally, you’ll have a chance to evaluate your progress with a quiz that’ll give you an idea of how much you’ve learned.

Free Bonus: Click here to get our free Python Cheat Sheet that shows you the basics of Python 3, like working with data types, dictionaries, lists, and Python functions.

Why You Should Use Python

Python, named after the British comedy group Monty Python, is a high-level, interpreted, interactive, and object-oriented programming language. Its flexibility allows you to do many things, both big and small. With Python, you can write basic programs and scripts and also to create complex and large-scale enterprise solutions. Here’s a sampling of its uses:

Building desktop applications, including GUI applications, CLI tools, and even games
Doing mathematical and scientific analysis of data
Building web and Internet applications
Doing computer systems administration and automating tasks
Performing DevOps tasks

You can find Python everywhere in the world of computer programming. For example, Python is the foundation of some of the world’s most popular websites, including Reddit, Dropbox, and YouTube, to name a few. The Python web framework Django powers both Instagram and Pinterest.

Python has a bunch of features that make it attractive as your first programming language:

Free: Python is available free of charge, even for commercial purposes.
Open source: Anyone can contribute to Python development.
Accessible: People of all ages, from school children to retirees, have learned Python, and so can you.
Versatile: Python can help you solve problems in many fields, including scripting, data science, web development, GUI development, and more.
Powerful: You can code small scripts to automate repetitive tasks, and you can also create complex and large-scale enterprise solutions with Python.

Compared to other programming languages, Python has the following features:

Interpreted: It’s portable and quicker to experiment with than compiled languages.
Multiparadigm: It lets you write code in different styles, including object-oriented, imperative, and functional style.
Dynamically typed: It checks variable types at runtime, so you don’t need to declare them explicitly.
Strongly typed: It won’t let unsafe operations on incompatible types go unnoticed.

There’s a lot more to learn about Python. But by now, you should have a better idea of why Python is so popular and why you should consider learning to program with it.

How to Download and Install Python

Python works on Linux, Mac, Windows, and several other platforms. It comes preinstalled on macOS and on most Linux distributions. However, if you want to be up to date, then you probably need to download and install the latest version. You also have the choice of using different Python versions in different projects if you want to.

To check what Python version has been installed globally in your operating system, open the terminal or command line and run the following command:

$ python3 -V

This command prints the version of your system’s default Python 3 installation. Note that you use python3 instead of python because some operating systems still include Python 2 as their default Python installation.

Installing Python From Binaries

Regardless of your operating system, you can download an appropriate version of Python from the official site. Go there and grab the appropriate 32-bit or 64-bit version for your operating system and processor.

Selecting and downloading a Python binary from the language’s official site is often a good choice. However, there are some OS-specific alternatives:

macOS: You have the option of installing Python from Homebrew.
Linux: You can install several Python versions using your distribution’s package manager.
Windows: You can install Python from the Microsoft Store.

You can also use the Anaconda distribution to install Python along with a rich set of packages and libraries, or you can use Miniconda if you want to install only the packages you need.

Read the full article at https://realpython.com/python-first-steps/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #44: Creating an Interactive Online Python Conference for PyCascades 2021

2021-01-22T12:00:00+00:00

How do you create a virtual conference that retains the interactivity of an in-person event? What are the tools needed for talk submissions, ticketing, and live hosting? Can you find those tools written in Python? This week on the show, we have several of the organizers of the PyCascades 2021 conference. They share the process of restructuring a Python conference to meet those challenges.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

C for Python Programmers

2021-01-20T14:00:00+00:00

The purpose of this tutorial is to get an experienced Python programmer up to speed with the basics of the C language and how it’s used in the CPython source code. It assumes you already have an intermediate understanding of Python syntax.

That said, C is a fairly limited language, and most of its usage in CPython falls under a small set of syntax rules. Getting to the point where you understand the code is a much smaller step than being able to write C effectively. This tutorial is aimed at the first goal but not the second.

In this tutorial, you’ll learn:

What the C preprocessor is and what role it plays in building C programs
How you can use preprocessor directives to manipulate source files
How C syntax compares to Python syntax
How to create loops, functions, strings, and other features in C

One of the first things that stands out as a big difference between Python and C is the C preprocessor. You’ll look at that first.

Note: This tutorial is adapted from the appendix, “Introduction to C for Python Programmers,” in CPython Internals: Your Guide to the Python Interpreter.

Free Download: Get a sample chapter from CPython Internals: Your Guide to the Python 3 Interpreter showing you how to unlock the inner workings of the Python language, compile the Python interpreter from source code, and participate in the development of CPython.

The C Preprocessor

The preprocessor, as the name suggests, is run on your source files before the compiler runs. It has very limited abilities, but you can use them to great advantage in building C programs.

The preprocessor produces a new file, which is what the compiler will actually process. All the commands to the preprocessor start at the beginning of a line, with a # symbol as the first non-whitespace character.

The main purpose of the preprocessor is to do text substitution in the source file, but it will also do some basic conditional code with #if or similar statements.

You’ll start with the most frequent preprocessor directive: #include.

`#include`

#include is used to pull the contents of one file into the current source file. There’s nothing sophisticated about #include. It reads a file from the file system, runs the preprocessor on that file, and puts the results into the output file. This is done recursively for each #include directive.

For example, if you look at CPython’s Modules/_multiprocessing/semaphore.c file, then near the top you’ll see the following line:

#include "multiprocessing.h"

This tells the preprocessor to pull in the entire contents of multiprocessing.h and put them into the output file at this position.

You’ll notice two different forms for the #include statement. One of them uses quotes ("") to specify the name of the include file, and the other uses angle brackets (<>). The difference comes from which paths are searched when looking for the file on the file system.

If you use <> for the filename, then the preprocessor will look only at system include files. Using quotes around the filename instead will force the preprocessor to look in the local directory first and then fall back to the system directories.

`#define`

#define allows you to do simple text substitution and also plays into the #if directives you’ll see below.

At its most basic, #define lets you define a new symbol that gets replaced with a text string in the preprocessor output.

Continuing in semphore.c, you’ll find this line:

#define SEM_FAILED NULL

This tells the preprocessor to replace every instance of SEM_FAILED below this point with the literal string NULL before the code is sent to the compiler.

Read the full article at https://realpython.com/c-for-python-programmers/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Introduction to Sorting Algorithms in Python

2021-01-19T14:00:00+00:00

Sorting is a basic building block that many other algorithms are built upon. It’s related to several exciting ideas that you’ll see throughout your programming career. Understanding how sorting algorithms in Python work behind the scenes is a fundamental step toward implementing correct and efficient algorithms that solve real-world problems.

In this course, you’ll learn:

How different sorting algorithms in Python work and how they compare under different circumstances
How Python’s built-in sort functionality works behind the scenes
How different computer science concepts like recursion and divide and conquer apply to sorting
What Big O notation is and how to use it to compare the efficiency of different algorithms

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Make Your First Python Game: Rock, Paper, Scissors!

2021-01-18T14:00:00+00:00

Game programming is a great way to learn how to program. You use many tools that you’ll see in the real world, plus you get to play a game to test your results! An ideal game to start your Python game programming journey is rock paper scissors.

In this tutorial, you’ll learn how to:

Code your own rock paper scissors game
Take in user input with input()
Play several games in a row using a while loop
Clean up your code with Enum and functions
Define more complex rules with a dictionary

Free Bonus: 5 Thoughts On Python Mastery, a free course for Python developers that shows you the roadmap and the mindset you’ll need to take your Python skills to the next level.

What Is Rock Paper Scissors?

You may have played rock paper scissors before. Maybe you’ve used it to decide who pays for dinner or who gets first choice of players for a team.

If you’re unfamiliar, rock paper scissors is a hand game for two or more players. Participants say “rock, paper, scissors” and then simultaneously form their hands into the shape of a rock (a fist), a piece of paper (palm facing downward), or a pair of scissors (two fingers extended). The rules are straightforward:

Rock smashes scissors.
Paper covers rock.
Scissors cut paper.

Now that you have the rules down, you can start thinking about how they might translate to Python code.

Play a Single Game of Rock Paper Scissors in Python

Using the description and rules above, you can make a game of rock paper scissors. Before you dive in, you’re going to need to import the module you’ll use to simulate the computer’s choices:

import random

Awesome! Now you’re able to use the different tools inside random to randomize the computer’s actions in the game. Now what? Since your users will also need to be able to choose their actions, the first logical thing you need is a way to take in user input.

Take User Input

Taking input from a user is pretty straightforward in Python. The goal here is to ask the user what they would like to choose as an action and then assign that choice to a variable:

user_action = input("Enter a choice (rock, paper, scissors): ")

This will prompt the user to enter a selection and save it to a variable for later use. Now that the user has selected an action, the computer needs to decide what to do.

Make the Computer Choose

A competitive game of rock paper scissors involves strategy. Rather than trying to develop a model for that, though, you can save yourself some time by having the computer select a random action. Random selections are a great way to have the computer choose a pseudorandom value.

You can use random.choice() to have the computer randomly select between the actions:

possible_actions = ["rock", "paper", "scissors"]
computer_action = random.choice(possible_actions)

This allows a random element to be selected from the list. You can also print the choices that the user and the computer made:

print(f"\nYou chose {user_action}, computer chose {computer_action}.\n")

Printing the user and computer actions can be helpful to the user, and it can also help you debug later on in case something isn’t quite right with the outcome.

Determine a Winner

Read the full article at https://realpython.com/python-rock-paper-scissors/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #43: Deep Reinforcement Learning in a Notebook With Jupylet + Gaming and Synthesis

2021-01-15T12:00:00+00:00

What is it like to design a Python library for three different audiences? This week on the show, we have Nir Aides, creator of Jupylet. His new library is designed for deep reinforcement learning researchers, musicians interested in live music coding, and kids interested in learning to program. Everything is designed to run inside of a Jupyter notebook.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Sentiment Analysis: First Steps With Python's NLTK Library

2021-01-13T14:00:00+00:00

Once you understand the basics of Python, familiarizing yourself with its most popular packages will not only boost your mastery over the language but also rapidly increase your versatility. In this tutorial, you’ll learn the amazing capabilities of the Natural Language Toolkit (NLTK) for processing and analyzing text, from basic functions to sentiment analysis powered by machine learning!

Sentiment analysis can help you determine the ratio of positive to negative engagements about a specific topic. You can analyze bodies of text, such as comments, tweets, and product reviews, to obtain insights from your audience. In this tutorial, you’ll learn the important features of NLTK for processing text data and the different approaches you can use to perform sentiment analysis on your data.

By the end of this tutorial, you’ll be ready to:

Split and filter text data in preparation for analysis
Analyze word frequency
Find concordance and collocations using different methods
Perform quick sentiment analysis with NLTK’s built-in classifier
Define features for custom classification
Use and compare classifiers for sentiment analysis with NLTK

Free Bonus: Click here to get our free Python Cheat Sheet that shows you the basics of Python 3, like working with data types, dictionaries, lists, and Python functions.

Getting Started With NLTK

The NLTK library contains various utilities that allow you to effectively manipulate and analyze linguistic data. Among its advanced features are text classifiers that you can use for many kinds of classification, including sentiment analysis.

Sentiment analysis is the practice of using algorithms to classify various samples of related text into overall positive and negative categories. With NLTK, you can employ these algorithms through powerful built-in machine learning operations to obtain insights from linguistic data.

Installing and Importing

You’ll begin by installing some prerequisites, including NLTK itself as well as specific resources you’ll need throughout this tutorial.

First, use pip to install NLTK:

$ python3 -m pip install nltk

While this will install the NLTK module, you’ll still need to obtain a few additional resources. Some of them are text samples, and others are data models that certain NLTK functions require.

To get the resources you’ll need, use nltk.download():

import nltk

nltk.download()

NLTK will display a download manager showing all available and installed resources. Here are the ones you’ll need to download for this tutorial:

names: A list of common English names compiled by Mark Kantrowitz
stopwords: A list of really common words, like articles, pronouns, prepositions, and conjunctions
state_union: A sample of transcribed State of the Union addresses by different US presidents, compiled by Kathleen Ahrens
twitter_samples: A list of social media phrases posted to Twitter
movie_reviews: Two thousand movie reviews categorized by Bo Pang and Lillian Lee
averaged_perceptron_tagger: A data model that NLTK uses to categorize words into their part of speech
vader_lexicon: A scored list of words and jargon that NLTK references when performing sentiment analysis, created by C.J. Hutto and Eric Gilbert
punkt: A data model created by Jan Strunk that NLTK uses to split full texts into word lists

Note: Throughout this tutorial, you’ll find many references to the word corpus and its plural form, corpora. A corpus is a large collection of related text samples. In the context of NLTK, corpora are compiled with features for natural language processing (NLP), such as categories and numerical scores for particular features.

A quick way to download specific resources directly from the console is to pass a list to nltk.download():

>>>

>>> import nltk

>>> nltk.download([
...     "names",
...     "stopwords",
...     "state_union",
...     "twitter_samples",
...     "movie_reviews",
...     "averaged_perceptron_tagger",
...     "vader_lexicon",
...     "punkt",
... ])
[nltk_data] Downloading package names to /home/user/nltk_data...
[nltk_data]   Unzipping corpora/names.zip.
[nltk_data] Downloading package stopwords to /home/user/nltk_data...
[nltk_data]   Unzipping corpora/stopwords.zip.
[nltk_data] Downloading package state_union to
[nltk_data]     /home/user/nltk_data...
[nltk_data]   Unzipping corpora/state_union.zip.
[nltk_data] Downloading package twitter_samples to
[nltk_data]     /home/user/nltk_data...
[nltk_data]   Unzipping corpora/twitter_samples.zip.
[nltk_data] Downloading package movie_reviews to
[nltk_data]     /home/user/nltk_data...
[nltk_data]   Unzipping corpora/movie_reviews.zip.
[nltk_data] Downloading package averaged_perceptron_tagger to
[nltk_data]     /home/user/nltk_data...
[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.
[nltk_data] Downloading package vader_lexicon to
[nltk_data]     /home/user/nltk_data...
[nltk_data] Downloading package punkt to /home/user/nltk_data...
[nltk_data]   Unzipping tokenizers/punkt.zip.
True

This will tell NLTK to find and download each resource based on its identifier.

Should NLTK require additional resources that you haven’t installed, you’ll see a helpful LookupError with details and instructions to download the resource:

>>>

>>> import nltk

>>> w = nltk.corpus.shakespeare.words()
...
LookupError:
**********************************************************************
  Resource shakespeare not found.
  Please use the NLTK Downloader to obtain the resource:

  >>> import nltk
  >>> nltk.download('shakespeare')
...

The LookupError specifies which resource is necessary for the requested operation along with instructions to download it using its identifier.

Compiling Data

Read the full article at https://realpython.com/python-nltk-sentiment-analysis/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Managing Python Dependencies

2021-01-12T14:00:00+00:00

Managing Python Dependencies is your “one-stop shop” for picking up modern Python dependency management practices and workflows with minimal time investment.

The course consists of 32 bite-sized video lessons, each focusing on a single concept. Progressing through the course, you’ll quickly build up a comprehensive knowledge of dependency management best practices in Python at your own, comfortable pace.

Along the way, you’ll see hands on examples and step-by-step workflows that reinforce the skills you’re learning.

By the end, you’ll know how to apply Python’s recommended dependency management tools, like pip, virtualenvs, and requirements files effectively in the most common day-to-day development scenarios on Linux, macOS, and Windows.

With Managing Python Dependencies you will:

Code at a higher level and become more efficient: Leveraging Python’s rich third-party library ecosystem let’s you write better programs in a shorter amount of time. With a large body of freely available modules you can avoid reinventing the wheel and deliver higher quality Python software, faster. This is a great way for you to demonstrate senior level skills that will benefit your development career.
Get up to speed with Python dependency management quickly: Your time is your most important asset. If you can use it efficiently you will save your sanity—and a nice stack of money. This course is concise but thorough and will help you attain solid Python dependency management knowledge fast.
Go from “writing scripts” to “building applications” with Python: By taking advantage of Python’s rich packaging ecosystem you’ll be able to build substantial and full-featured applications in Python. You’ll know the best practices for finding and documenting application dependencies that put you right on track for deploying and shipping production-grade apps.
Find great libraries for a specific task at hand: This course teaches you a unique 7-step workflow for finding and identifying quality Python packages. Find out which libraries and tools are available on PyPI to help you. By quickly and easily identifying the correct libraries you’ll speed up your development efficiency by a large margin. Meet your deadlines and write better code at the same time by not having to “reinvent the wheel.”
Master “the tools of the trade” for dependency management: With this course you’ll master the tools and workflows recommended by the official Python Packaging Authority. By getting those skills under your belt you’ll be ready to work with the Python development and production environments used by professional development teams across the world. Knowing these “tools of the trade” by heart puts you at an advantage in any job interview situation.
“Productionize” your projects and share them with the world: You’ll see how to apply the best practices for defining and installing package dependencies in Python. You’ll know how to get your programs ready to be deployed on production and automated testing environments and how to make it easy for other developers to contribute code with minimal setup effort.
Become more valuable as an employee and team member: With my detailed 7-step workflow for researching quality Python packages you’ll know how to document and justify added program dependencies to your team and your manager. By taking on more responsibilities and picking up these senior-level “architectural” skills you’ll rise head and shoulders above other devs stuck at the “code monkey” level.

Who Is This Course For?

This course is for Python developers wanting to break through to the next phase of developing code by becoming more efficient, productive, and skilled using Python’s rich library ecosystem.

If you’ve ever caught yourself thinking “There’s got to be a Python package out there that does exactly what I want…But how do I find it?” this course will fill in the missing pieces for you.

Discover the industry best practices around choosing and managing third-party dependencies for your Python 2 or Python 3 projects on Windows, macOS, and Linux.

If you already know how to use alternative package managers like Conda you’ll discover how to use the standards-compliant tools and workflows supported by any Python distribution and used in most production application deployments.

Course Goals

By the end of the course you’ll know how to:

Install, use, and manage third-party Python packages with the “pip” package manager on Windows, macOS, and Linux.
Isolate project dependencies with so-called virtual environments to avoid version conflicts in your Python projects.
Apply a complete 7-step workflow for finding and identifying quality third-party packages to use in your own Python projects (and justifying your decisions to your team or manager.)
Set up repeatable development environments and application deployments using the “pip” package manager and requirements files.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

NumPy Tutorial: Your First Steps Into Data Science in Python

2021-01-11T14:00:00+00:00

NumPy is a Python library that provides a simple yet powerful data structure: the n-dimensional array. This is the foundation on which almost all the power of Python’s data science toolkit is built, and learning NumPy is the first step on any Python data scientist’s journey. This tutorial will provide you with the knowledge you need to use NumPy and the higher-level libraries that rely on it.

In this tutorial you’ll learn:

What core concepts in data science are made possible by NumPy
How to create NumPy arrays using various methods
How to manipulate NumPy arrays to perform useful calculations
How to apply these new skills to real-world problems

To get the most out of this NumPy tutorial, you should be familiar with writing Python code. Working through the Introduction to Python learning path is a great way to make sure you’ve got the basic skills covered. If you’re familiar with matrix mathematics, then that will certainly be helpful as well. You don’t need to know anything about data science, however. You’ll learn that here.

There’s also a repository of NumPy code samples that you’ll see throughout this tutorial. You can use it for reference and experiment with the examples to see how changing the code changes the outcome. To download the code, click the link below:

Get Sample Code: Click here to get the sample code you’ll use to learn about NumPy in this tutorial.

Choosing NumPy: The Benefits

Since you already know Python, you may be asking yourself if you really have to learn a whole new paradigm to do data science. Python’s for loops are awesome! Reading and writing CSV files can be done with traditional code. However, there are some convincing arguments for learning a new paradigm.

Here are the top four benefits that NumPy can bring to your code:

More speed: NumPy uses algorithms written in C that complete in nanoseconds rather than seconds.
Fewer loops: NumPy helps you to reduce loops and keep from getting tangled up in iteration indices.
Clearer code: Without loops, your code will look more like the equations you’re trying to calculate.
Better quality: There are thousands of contributors working to keep NumPy fast, friendly, and bug free.

Because of these benefits, NumPy is the de facto standard for multidimensional arrays in Python data science, and many of the most popular libraries are built on top of it. Learning NumPy is a great way to set down a solid foundation as you expand your knowledge into more specific areas of data science.

Installing NumPy

It’s time to get everything set up so you can start learning how to work with NumPy. There are a few different ways to do this, and you can’t go wrong by following the instructions on the NumPy website. But there are some extra details to be aware of that are outlined below.

You’ll also be installing Matplotlib. You’ll use it in one of the later examples to explore how other libraries make use of NumPy.

Using Repl.it as an Online Editor

If you just want to get started with some examples, follow along with this tutorial, and start building some muscle memory with NumPy, then Repl.it is a great option for in-browser editing. You can sign up and fire up a Python environment in minutes. Along the left side, there’s a tab for packages. You can add as many as you want. For this NumPy tutorial, go with the current versions of NumPy and Matplotlib.

Here’s where you can find the packages in the interface:

Luckily, they allow you to just click and install.

Installing NumPy With Anaconda

The Anaconda distribution is a suite of common Python data science tools bundled around a package manager that helps manage your virtual environments and project dependencies. It’s built around conda, which is the actual package manager. This is the method recommended by the NumPy project, especially if you’re stepping into data science in Python without having already set up a complex development environment.

If you’ve already got a workflow you like that uses pip, Pipenv, Poetry, or some other toolset, then it might be better not to add conda to the mix. The conda package repository is separate from PyPI, and conda itself sets up a separate little island of packages on your machine, so managing paths and remembering which package lives where can be a nightmare.

Once you’ve got conda installed, you can run the install command for the libraries you’ll need:

$ conda install numpy matplotlib

This will install what you need for this NumPy tutorial, and you’ll be all set to go.

Read the full article at https://realpython.com/numpy-tutorial/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #42: What Is Data Engineering and Researching 10 Million Jupyter Notebooks

2021-01-08T12:00:00+00:00

Are you familiar with the role data engineers play in the modern landscape of data science and Python? Data engineering is a sub-discipline that focuses on the transportation, transformation, and storage of data. This week on the show, David Amos is back, and he's brought another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Develop Data Visualization Interfaces in Python With Dash

2021-01-06T14:00:00+00:00

In the past, creating analytical web applications was a task for seasoned developers that required knowledge of multiple programming languages and frameworks. That’s no longer the case. Nowadays, you can make data visualization interfaces using pure Python. One popular tool for this is Dash.

Dash gives data scientists the ability to showcase their results in interactive web applications. You don’t need to be an expert in web development. In an afternoon, you can build and deploy a Dash app to share with others.

In this tutorial, you’ll learn how to:

Create a Dash application
Use Dash core components and HTML components
Customize the style of your Dash application
Use callbacks to build interactive applications
Deploy your application on Heroku

You can download the source code, data, and resources for the sample application you’ll make in this tutorial by clicking the link below:

Get the Source Code: Click here to get the source code you’ll use to learn about creating data visualization interfaces in Python with Dash in this tutorial.

What Is Dash?

Dash is an open source framework for building data visualization interfaces. Released in 2017 as a Python library, it’s grown to include implementations for R and Julia. Dash helps data scientists build analytical web applications without requiring advanced web development knowledge.

Three technologies constitute the core of Dash:

Flask supplies the web server functionality.
React.js renders the user interface of the web page.
Plotly.js generates the charts used in your application.

But you don’t have to worry about making all these technologies work together. Dash will do that for you. You just need to write Python, R, or Julia and sprinkle it with a bit of CSS.

Plotly, a Canada-based company, built Dash and supports its development. You may know the company from the popular graphing libraries that share its name. Plotly (the company) open-sourced Dash and released it under an MIT license, so you can use Dash at no cost.

Plotly also offers a commercial companion to Dash called Dash Enterprise. This paid service provides companies with support services such as hosting, deploying, and handling authentication on Dash applications. But these features live outside of Dash’s open source ecosystem.

Dash will help you build dashboards quickly. If you’re used to analyzing data or building data visualizations using Python, then Dash will be a useful addition to your toolbox. Here are a few examples of what you can make with Dash:

This is just a tiny sample. If you’d like to see other interesting use cases, then go check the Dash App Gallery.

Note: You don’t need advanced knowledge of web development to follow this tutorial, but some familiarity with HTML and CSS won’t hurt.

The rest of this tutorial assumes you know the basics of the following topics:

Python graphing libraries such as Plotly, Bokeh, or Matplotlib
HTML and the structure of an HTML file
CSS and style sheets

If you feel comfortable with the requirements and want to learn how to use Dash in your next project, then continue to the following section!

Get Started With Dash in Python

In this tutorial, you’ll go through the end-to-end process of building a dashboard using Dash. If you follow along with the examples, then you’ll go from a bare-bones dashboard on your local machine to a styled dashboard deployed on Heroku.

To build the dashboard, you’ll use a dataset of sales and prices of avocados in the United States between 2015 and 2018. This dataset was compiled by Justin Kiggins using data from the Hass Avocado Board.

How to Set Up Your Local Environment

To develop your app, you’ll need a new directory to store your code and data and a clean Python 3 virtual environment. To create those, follow the instructions below, choosing the version that matches your operating system.

If you’re using Windows, then open a command prompt and execute these commands:

c:\> mkdir avocado_analytics && cd avocado_analytics
c:\> c:\path\to\python\launcher\python -m venv venv
c:\> venv\Scripts\activate.bat

Read the full article at https://realpython.com/python-dash/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Building With Django REST Framework

2021-01-05T14:00:00+00:00

REST is a loosely defined protocol for listing, creating, changing, and deleting data on your server over HTTP. The Django REST framework (DRF) is a toolkit built on top of the Django web framework that reduces the amount of code you need to write to create REST HTTP API interfaces.

In this course you’ll learn about:

The REST protocol
DRF Serializers and how to use them with Django objects
Using Django views and DRF ViewSet classes to create REST end-points
Multiple flavors of renderers and how to control their output
Specifying permissions and limiting who can see what data in your REST API

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python's .append(): Add Items to Your Lists in Place

2021-01-04T14:00:00+00:00

Adding items to a list is a fairly common task in Python, so the language provides a bunch of methods and operators that can help you out with this operation. One of those methods is .append(). With .append(), you can add items to the end of an existing list object. You can also use .append() in a for loop to populate lists programmatically.

In this tutorial, you’ll learn how to:

Work with .append()
Populate lists using .append() and a for loop
Replace .append() with list comprehensions
Work with .append() in array.array() and collections.deque()

You’ll also code some examples of how to use .append() in practice. With this knowledge, you’ll be able to effectively use .append() in your programs.

Free Download: Get a sample chapter from Python Basics: A Practical Introduction to Python 3 to see how you can go from beginner to intermediate in Python with a complete curriculum, up-to-date for Python 3.8.

Adding Items to a List With Python’s `.append()`

Python’s .append() takes an object as an argument and adds it to the end of an existing list, right after its last element:

>>>

>>> numbers = [1, 2, 3]
>>> numbers.append(4)
>>> numbers
[1, 2, 3, 4]

Every time you call .append() on an existing list, the method adds a new item to the end, or right side, of the list. The following diagram illustrates the process:

Python lists reserve extra space for new items at the end of the list. A call to .append() will place new items in the available space.

In practice, you can use .append() to add any kind of object to a given list:

>>>

>>> mixed = [1, 2]

>>> mixed.append(3)
>>> mixed
[1, 2, 3]

>>> mixed.append("four")
>>> mixed
[1, 2, 3, 'four']

>>> mixed.append(5.0)
>>> mixed
[1, 2, 3, 'four', 5.0]

Lists are sequences that can hold different data types and Python objects, so you can use .append() to add any object to a given list. In this example, you first add an integer number, then a string, and finally a floating-point number. However, you can also add another list, a dictionary, a tuple, a user-defined object, and so on.

Using .append() is equivalent to the following operation:

>>>

>>> numbers = [1, 2, 3]

>>> # Equivalent to numbers.append(4)
>>> numbers[len(numbers):] = [4]
>>> numbers
[1, 2, 3, 4]

In the highlighted line, you perform two operations at the same time:

You take a slice from numbers using the expression numbers[len(numbers):].
You assign an iterable to that slice.

The slicing operation takes the space after the last item in numbers. Meanwhile, the assignment operation unpacks the items in the list to the right of the assignment operator and adds them to numbers. However, there’s an important difference between using this kind of assignment and using .append(). With the assignment, you can add several items to the end of your list at once:

>>>

>>> numbers = [1, 2, 3]

>>> numbers[len(numbers):] = [4, 5, 6]
>>> numbers
[1, 2, 3, 4, 5, 6]

In this example, the highlighted line takes a slice from the end of numbers, unpacks the items in the list on the right side, and adds them to the slice as individual items.

`.append()` Adds a Single Item

With .append(), you can add a number, list, tuple, dictionary, user-defined object, or any other object to an existing list. However, you need to keep in mind that .append() adds only a single item or object at a time:

>>>

>>> x = [1, 2, 3, 4]
>>> y = (5, 6)

>>> x.append(y)
>>> x
[1, 2, 3, 4, (5, 6)]

What happens here is that .append() adds the tuple object y to the end of your target list, x. What if you want to add each item in y to the end of x as an individual item and get [1, 2, 3, 4, 5, 6]? In that case, you can use .extend():

Read the full article at https://realpython.com/python-append/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Django Admin Customization

2020-12-29T14:00:00+00:00

The Django framework comes with a powerful administrative tool called admin. You can use it out of the box to quickly add, delete, or edit any database model from a web interface. But with a little extra code, you can customize the Django admin to take your admin capabilities to the next level.

In this course, you’ll learn how to:

Add attribute columns in the model object list
Link between model objects
Add filters to the model object list
Make model object lists searchable
Modify the object edit forms
Override Django admin templates

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python and MySQL Database: A Practical Introduction

2020-12-28T14:00:00+00:00

MySQL is one of the most popular database management systems (DBMSs) on the market today. It ranked second only to the Oracle DBMS in this year’s DB-Engines Ranking. As most software applications need to interact with data in some form, programming languages like Python provide tools for storing and accessing these data sources.

Using the techniques discussed in this tutorial, you’ll be able to efficiently integrate a MySQL database with a Python application. You’ll develop a small MySQL database for a movie rating system and learn how to query it directly from your Python code.

By the end of this tutorial, you’ll be able to:

Identify unique features of MySQL
Connect your application to a MySQL database
Query the database to fetch required data
Handle exceptions that occur while accessing the database
Use best practices while building database applications

To get the most out of this tutorial, you should have a working knowledge of Python concepts like for loops, functions, exception handling, and installing Python packages using pip. You should also have a basic understanding of relational database management systems and SQL queries like SELECT, DROP, CREATE, and JOIN.

Free Download: Get a sample chapter from Python Tricks: The Book that shows you Python’s best practices with simple examples you can apply instantly to write more beautiful + Pythonic code.

Comparing MySQL to Other SQL Databases

SQL stands for Structured Query Language and is a widely used programming language for managing relational databases. You may have heard of the different flavors of SQL-based DBMSs. The most popular ones include MySQL, PostgreSQL, SQLite, and SQL Server. All of these databases are compliant with the SQL standards but with varying degrees of compliance.

Being open source since its inception in 1995, MySQL quickly became a market leader among SQL solutions. MySQL is also a part of the Oracle ecosystem. While its core functionality is completely free, there are some paid add-ons as well. Currently, MySQL is used by all major tech firms, including Google, LinkedIn, Uber, Netflix, Twitter, and others.

Apart from a large open source community for support, there are many other reasons for MySQL’s success:

Ease of installation: MySQL was designed to be user-friendly. It’s quite straightforward to set up a MySQL database, and several widely available third-party tools, like phpMyAdmin, further streamline the setup process. MySQL is available for all major operating systems, including Windows, macOS, Linux, and Solaris.
Speed: MySQL holds a reputation for being an exceedingly fast database solution. It has a relatively smaller footprint and is extremely scalable in the long run.
User privileges and security: MySQL comes with a script that allows you to set the password security level, assign admin passwords, and add and remove user account privileges. This script uncomplicates the admin process for a web hosting user management portal. Other DBMSs, like PostgreSQL, use config files that are more complicated to use.

While MySQL is famous for its speed and ease of use, you can get more advanced features with PostgreSQL. Also, MySQL isn’t fully SQL compliant and has certain functional limitations, like no support for FULL JOIN clauses.

You might also face some issues with concurrent reading and writing in MySQL. If your software has many users writing data to it at once, then PostgreSQL might be a more suitable choice.

Note: For a more in-depth comparison of MySQL and PostgreSQL in a real-world context, check out Why Uber Engineering Switched from Postgres to MySQL.

SQL Server is also a very popular DBMS and is known for its reliability, efficiency, and security. It’s preferred by companies, especially in the banking domain, who regularly deal with large traffic workloads. It’s a commercial solution and is one of the systems that are most compatible with Windows services.

In 2010, when Oracle acquired Sun Microsystems and MySQL, many were worried about MySQL’s future. At the time, Oracle was MySQL’s biggest competitor. Developers feared that this was a hostile takeover from Oracle with the aim of destroying MySQL.

Several developers led by Michael Widenius, the original author of MySQL, created a fork of the MySQL code base and laid the foundation of MariaDB. The aim was to secure access to MySQL and keep it free forever.

To date, MariaDB remains fully GPL licensed, keeping it completely in the public domain. Some features of MySQL, on the other hand, are available only with paid licenses. Also, MariaDB provides several extremely useful features that aren’t supported by MySQL server, like distributed SQL and columnar storage. You can find more differences between MySQL and MariaDB listed on MariaDB’s website.

MySQL uses a very similar syntax to the Standard SQL. There are, however, some notable differences mentioned in the official documentation.

Installing MySQL Server and MySQL Connector/Python

Now, to start working through this tutorial, you need to set up two things: a MySQL server and a MySQL connector. MySQL server will provide all the services required for handling your database. Once the server is up and running, you can connect your Python application with it using MySQL Connector/Python.

Installing MySQL Server

The official documentation details the recommended way to download and install MySQL server. You’ll find instructions for all popular operating systems, including Windows, macOS, Solaris, Linux, and many more.

For Windows, the best way is to download MySQL Installer and let it take care of the entire process. The installation manager also helps you configure the security settings of the MySQL server. On the Accounts and Roles page, you need to enter a password for the root (admin) account and also optionally add other users with varying privileges:

MySQL Installer Account Setup

Read the full article at https://realpython.com/python-mysql/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #41: 2020 Real Python Articles in Review

2020-12-25T12:00:00+00:00

It's been quite the year! The Real Python team has written, edited, curated, illustrated, and produced a mountain of Python articles this year. We also upgraded the site and membership with office hours, transcripts, this podcast, and much more. We are joined by two members of the Real Python team, David Amos and Joanna Jablonski. We wanted to share a year-end wrap-up with a collection of articles that showcase a diversity of Python topics and the quality of what our team created this year.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Serializing Objects With the Python pickle Module

2020-12-22T14:00:00+00:00

As a developer, you may sometimes need to send complex object hierarchies over a network or save the internal state of your objects to a disk or database for later use. To accomplish this, you can use a process called serialization, which is fully supported by the standard library thanks to the Python pickle module.

In this course, you’ll learn:

What it means to serialize and deserialize an object
Which modules you can use to serialize objects in Python
Which kinds of objects can be serialized with the Python pickle module
How to use the Python pickle module to serialize object hierarchies
What the risks are when deserializing an object from an untrusted source

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Use PyQt's QThread to Prevent Freezing GUIs

2020-12-21T14:00:00+00:00

PyQt graphical user interface (GUI) applications have a main thread of execution that runs the event loop and GUI. If you launch a long-running task in this thread, then your GUI will freeze until the task terminates. During that time, the user won’t be able to interact with the application, resulting in a bad user experience. Luckily, PyQt’s QThread class allows you to work around this issue.

In this tutorial, you’ll learn how to:

Use PyQt’s QThread to prevent freezing GUIs
Create reusable threads with QThreadPool and QRunnable
Manage interthread communication using signals and slots
Safely use shared resources with PyQt’s locks
Use best practices for developing GUI applications with PyQt’s thread support

For a better understanding of how to use PyQt’s threads, some previous knowledge of GUI programming with PyQt and Python multithreaded programming would be helpful.

Free Bonus: Get a sample chapter from Python Basics: A Practical Introduction to Python 3 to see how you can go from beginner to intermediate in Python with a complete curriculum, up-to-date for Python 3.8.

Freezing a GUI With Long-Running Tasks

Long-running tasks occupying the main thread of a GUI application and causing the application to freeze is a common issue in GUI programming that almost always results in a bad user experience. For example, consider the following GUI application:

Say you need the Counting label to reflect the total number of clicks on the Click me! button. Clicking the Long-Running Task! button will launch a task that takes a lot of time to finish. Your long-running task could be a file download, a query to a large database, or any other resource-intensive operation.

Here’s a first approach to coding this application using PyQt and a single thread of execution:

import sys
from time import sleep

from PyQt5.QtCore import Qt
from PyQt5.QtWidgets import (
    QApplication,
    QLabel,
    QMainWindow,
    QPushButton,
    QVBoxLayout,
    QWidget,
)

class Window(QMainWindow):
    def __init__(self, parent=None):
        super().__init__(parent)
        self.clicksCount = 0
        self.setupUi()

    def setupUi(self):
        self.setWindowTitle("Freezing GUI")
        self.resize(300, 150)
        self.centralWidget = QWidget()
        self.setCentralWidget(self.centralWidget)
        # Create and connect widgets
        self.clicksLabel = QLabel("Counting: 0 clicks", self)
        self.clicksLabel.setAlignment(Qt.AlignHCenter | Qt.AlignVCenter)
        self.stepLabel = QLabel("Long-Running Step: 0")
        self.stepLabel.setAlignment(Qt.AlignHCenter | Qt.AlignVCenter)
        self.countBtn = QPushButton("Click me!", self)
        self.countBtn.clicked.connect(self.countClicks)
        self.longRunningBtn = QPushButton("Long-Running Task!", self)
        self.longRunningBtn.clicked.connect(self.runLongTask)
        # Set the layout
        layout = QVBoxLayout()
        layout.addWidget(self.clicksLabel)
        layout.addWidget(self.countBtn)
        layout.addStretch()
        layout.addWidget(self.stepLabel)
        layout.addWidget(self.longRunningBtn)
        self.centralWidget.setLayout(layout)

    def countClicks(self):
        self.clicksCount += 1
        self.clicksLabel.setText(f"Counting: {self.clicksCount} clicks")

    def reportProgress(self, n):
        self.stepLabel.setText(f"Long-Running Step: {n}")

    def runLongTask(self):
        """Long-running task in 5 steps."""
        for i in range(5):
            sleep(1)
            self.reportProgress(i + 1)

app = QApplication(sys.argv)
win = Window()
win.show()
sys.exit(app.exec())

In this Freezing GUI application, .setupUi() creates all the required graphical components for the GUI. A click on the Click me! button calls .countClicks(), which makes the text of the Counting label reflect the number of button clicks.

Note: PyQt was first developed to target Python 2, which has an exec keyword. To avoid a name conflict on those earlier versions of PyQt, an underscore was added to the end of .exec_().

Even though PyQt5 targets only Python 3, which doesn’t have an exec keyword, the library provides two methods to start an application’s event loop:

.exec_()
.exec()

Both variations of the method work the same, so you can use either one in your applications.

Clicking the Long-Running Task! button calls .runLongTask(), which performs a task that takes 5 seconds to complete. This is a hypothetical task that you coded using time.sleep(secs), which suspends the execution of the calling thread for the given number of seconds, secs.

In .runLongTask(), you also call .reportProgress() to make the Long-Running Step label reflect the progress of the operation.

Does this application work as you intend? Run the application and check out its behavior:

When you click the Click me! button, the label shows the number of clicks. However, if you click the Long-Running Task! button, then the application becomes frozen and unresponsive. The buttons no longer respond to clicks and the labels don’t reflect the application’s state.

After five seconds, the application’s GUI gets updated again. The Counting label shows ten clicks, reflecting five clicks that occurred while the GUI was frozen. The Long-Running Step label doesn’t reflect the progress of your long-running operation. It jumps from zero to five without showing the intermediate steps.

Note: Even though your application’s GUI freezes during the long-running task, the application still registers events such as clicks and keystrokes. It’s just unable to process them until the main thread gets released.

The application’s GUI freezes as a result of a blocked main thread. The main thread is busy processing a long-running task and doesn’t immediately respond to the user’s actions. This is an annoying behavior because the user doesn’t know for sure if the application is working correctly or if it’s crashed.

Fortunately, there are some techniques you can use to work around this issue. A commonly used solution is to run your long-running task outside of the application’s main thread using a worker thread.

In the sections below, you’ll learn how to use PyQt’s built-in thread support to solve the issue of unresponsive or frozen GUIs and provide the best possible user experience in your applications.

Multithreading: The Basics

Sometimes you can divide your programs into several smaller subprograms, or tasks, that you can run in several threads. This might make your programs faster, or it might help you improve the user experience by preventing your programs from freezing while executing long-running tasks.

Read the full article at https://realpython.com/python-pyqt-qthread/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #40: How Python Manages Memory and Creating Arrays With np.linspace

2020-12-18T12:00:00+00:00

Have you wondered how Python manages memory? How are your variables stored in memory, and when do they get deleted? This week on the show, David Amos is here, and he has brought another batch of PyCoder's Weekly articles and projects. Along with the Real Python article on Python memory management, we also talk about another article about creating even and non-even spaced arrays in Python with np.linspace.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python Turtle for Beginners

2020-12-15T14:00:00+00:00

In this step-by-step course, you’ll learn the basics of Python programming with the help of a simple and interactive Python library called turtle. If you’re a beginner to Python, then this course will definitely help you on your journey as you take your first steps into the world of programming. The Python turtle library comes with a similar interactive feature that gives new programmers a taste of what it’s like to work with Python.

In this course, you will:

Understand what the Python turtle library is
Learn how to set turtle up on your computer
Program with the Python turtle library
Grasp some important Python concepts and turtle commands
Develop a short but entertaining game using what you’ve learned

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #39: Generators, Coroutines, and Learning Python Through Exercises

2020-12-11T12:00:00+00:00

Have you started to use generators in Python? Are you unsure why you would even use one over a regular function? How do you use the special "send" method and the "yield from" syntax? This week on the show, we have Reuven Lerner to talk about his PyCon Africa 2020 talk titled "Generators, coroutines, and nanoservices."

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Speed Up Python With Concurrency

2020-12-08T14:00:00+00:00

Concurrency is the act of having your computer do multiple things at the same time. If you’ve heard lots of talk about asyncio being added to Python but are curious how it compares to other concurrency methods or are wondering what concurrency is and how it might speed up your program, you’ve come to the right place.

In this course, you’ll learn the following:

How I/O bound programs are effected by latency
Which concurrent programming patterns to use
What the differences are between the Python concurrency libraries
How to write code that uses the threading, asyncio, and multiprocessing libraries

Sample code was tested using Python 3.8.5. Much of the asyncio library has been in flux since Python 3.4, it is recommended to use at least Python 3.7 for the asyncio portions of the course.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #38: Looping With enumerate() and Python GUIs With PyQt

2020-12-04T12:00:00+00:00

If you're coming to Python from a different language, you may not know about a useful tool for working with loops, Python's built-in enumerate function. This week on the show, David Amos is here, and he has brought another batch of PyCoder's Weekly articles and projects. Along with the Real Python article covering the details of the enumerate function, we also talk about another article about constructing Python graphical user interface elements in PyQt.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

How Python Manages Memory

2020-12-01T14:00:00+00:00

Ever wonder how Python handles your data behind the scenes? How are your variables stored in memory? When do they get deleted?

In this course, we’re going to do a deep dive into the internals of Python to understand how it handles memory management.

By the end of this course, you’ll:

Learn more about low-level computing, specifically as relates to memory
Understand how Python abstracts lower-level operations
Learn about Python’s internal memory management algorithms

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #37: Teaching Python and Finding Resources for Students

2020-11-27T12:00:00+00:00

One of the best ways to learn something well is to teach it. This week on the show, we have Kelly Schuster-Paredes and Sean Tibor from the Teaching Python podcast. Sean and Kelly teach middle school students Python and share their art and science of teaching Python on their podcast. They wanted to come on the show to talk about the Real Python articles, quizzes, and other resources they use when teaching their students.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Formatting Python Strings

2020-11-24T14:00:00+00:00

In this course, you’ll see two items to add to your Python string formatting toolkit. You’ll learn about Python’s string format method and the formatted string literal, or f-string. You’ll learn about these formatting techniques in detail and add them to your Python string formatting toolkit.

In this course, you’ll learn about:

The string .format() method
The formatted string literal, or f-string

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #36: Sentiment Analysis, Fourier Transforms, and More Python Data Science

2020-11-20T12:00:00+00:00

Are you interested in learning more about Natural Language Processing? Have you heard of sentiment analysis? This week on the show, Kyle Stratis returns to talk about his new article titled, Use Sentiment Analysis With Python to Classify Movie Reviews. David Amos is also here, and all of us cover another batch of PyCoder’s Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #35: Security and Authorization in Your Python Web Applications

2020-11-13T12:00:00+00:00

So you built a web application in Python. Now how are you going to authorize users? Security goes beyond authentication. Who gets to do what, where, and when? This week on the show, we have Sam Scott, chief technology officer from Oso. Oso is an open-source policy engine for authorization that you embed in your application.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Real Python

Stochastic Gradient Descent Algorithm With Python and NumPy

Basic Gradient Descent Algorithm

Cost Function: The Goal of Optimization

Gradient of a Function: Calculus Refresher

Evaluate Expressions Dynamically With Python eval()

How to Use Python: Your First Steps

Why You Should Use Python

How to Download and Install Python

Installing Python From Binaries

The Real Python Podcast – Episode #44: Creating an Interactive Online Python Conference for PyCascades 2021

C for Python Programmers

The C Preprocessor

#include

#define

Introduction to Sorting Algorithms in Python

Make Your First Python Game: Rock, Paper, Scissors!

What Is Rock Paper Scissors?

Play a Single Game of Rock Paper Scissors in Python

Take User Input

Make the Computer Choose

Determine a Winner

The Real Python Podcast – Episode #43: Deep Reinforcement Learning in a Notebook With Jupylet + Gaming and Synthesis

Sentiment Analysis: First Steps With Python's NLTK Library

Getting Started With NLTK

Installing and Importing

Compiling Data

Managing Python Dependencies

Who Is This Course For?

Course Goals

NumPy Tutorial: Your First Steps Into Data Science in Python

Choosing NumPy: The Benefits

Installing NumPy

Using Repl.it as an Online Editor

Installing NumPy With Anaconda

The Real Python Podcast – Episode #42: What Is Data Engineering and Researching 10 Million Jupyter Notebooks

Develop Data Visualization Interfaces in Python With Dash

What Is Dash?

Get Started With Dash in Python

How to Set Up Your Local Environment

Building With Django REST Framework

Python's .append(): Add Items to Your Lists in Place

Adding Items to a List With Python’s .append()

.append() Adds a Single Item

Django Admin Customization

Python and MySQL Database: A Practical Introduction

Comparing MySQL to Other SQL Databases

Installing MySQL Server and MySQL Connector/Python

Installing MySQL Server

The Real Python Podcast – Episode #41: 2020 Real Python Articles in Review

Serializing Objects With the Python pickle Module

Use PyQt's QThread to Prevent Freezing GUIs

Freezing a GUI With Long-Running Tasks

Multithreading: The Basics

The Real Python Podcast – Episode #40: How Python Manages Memory and Creating Arrays With np.linspace

Python Turtle for Beginners

The Real Python Podcast – Episode #39: Generators, Coroutines, and Learning Python Through Exercises

Speed Up Python With Concurrency

The Real Python Podcast – Episode #38: Looping With enumerate() and Python GUIs With PyQt

How Python Manages Memory

The Real Python Podcast – Episode #37: Teaching Python and Finding Resources for Students

Formatting Python Strings

The Real Python Podcast – Episode #36: Sentiment Analysis, Fourier Transforms, and More Python Data Science

The Real Python Podcast – Episode #35: Security and Authorization in Your Python Web Applications

`#include`

`#define`

Adding Items to a List With Python’s `.append()`

`.append()` Adds a Single Item