Crack open the source code of any contemporary data journalism project, and you'll probably find some Python. The popular programming language is useful in a wide variety of journalism-related practices. From data cleaning and data analysis, to web scraping such as ProPublica's collection of Cook County Jail Data to web app development like the Baltimore Sun Public Salary Records project.
Python 3 in macOS Catalina: Fixing the abort trap. At some point in the beta program of macOS Catalina Homebrew ’s python 3 broke and only ended up showing an “Abort trap: 6” for every command that involved using it. This included pip3 and other tools that were previously downloaded and worked as expected.
Python's simple, readable syntax and active, open-source community make it a great first language for anyone who wants to learn to code.
- 'The basic premise of all Python development is to never use the system Python. You do not want the Mac OS X 'default Python' to be 'python3.' You want to never care about default Python.' How do we stop caring about the default? Moshe recommends using pyenv to manage Python environments (for a deeper dive on configuring pyenv, see this article.
- The macOS 10.15 SDK provides support for developing apps for Macs running macOS Catalina 10.15. The SDK comes bundled with Xcode 11 available from the Mac App Store. For information on the compatibility requirements for Xcode 11, see Xcode 11 Release Notes.
But how do you get Python? The process of setting everything up on your own desktop or laptop can get tricky, especially for newcomers to coding.
That's why I've put together this guide for anyone who wants to set up a Python development environment on their Mac.
Our aim is to strip the setup process down to the most essential steps and explain what's happening and why it's necessary. We'll also point out when you have the opportunity to further customize your development environment.
Without further delay, buckle up and let's get started!
A note to Windows users...
We're not ignoring you. We know you're out there, and we think you're awesome. You also deserve a slick Python setup, one that affords collaboration with your peers who've made different hardware choices. We'll be back soon with a similar how-to guide fit for you.
Step 0: Install any outstanding security/system updates
First, check your App Store icon for any red badges. If any of these nagging notifications are related to system or security updates, time to finally deal with them. It's OK. We all ignore these longer than we should.
Step 1: Open Terminal
All Macs ship with a built-in terminal emulator that's simply called 'Terminal'.
You'll find Terminal in your Applications folder under Utilities. You can also find it via the Spotlight search tool (type ⌘ + Spacebar, then start typing 'terminal').
This application is a throwback to a time before personal computers when people interacted with giant central mainframes via a simple hardware device, little more than a keyboard with a video display.
Like those early terminal devices, the Terminal application allows access to the command line interfaces (CLIs) of software installed on your computer. These are the advanced controls hidden behind the graphical user interfaces (GUIs) we normally use.
We can't point, click, swipe or otherwise work the command line in all the intuitive ways most of us prefer to use computers. Instead we type commands, then press RETURN, then the screen prints output (sometimes).
The methods for installing Python and setting up Python projects outlined below are available only via the command line. We'll provide the exact the commands you will need to execute, along with the proper arguments and options. You're encouraged to copy this input from the code snippet boxes, like this one:
Then paste them into your Terminal window. Also, don't forget to press RETURN in order to execute each command.
Go deeper...
Terminal will definitely get the job done. However, as you spend more time on the command line, you may want to liven things by changing your theme or otherwise customizing the experience.
If the range of customizations for Apple's Terminal (located under Preferences) feels too limited, you can switch to a third-party terminal emulator. Here are a couple of great, free options:
- iTerm2 has been the most popular choice for years, and it keeps getting better with each release.
- Hyper is the latest hotness. It's dazzling. It's extensible. And because it's built with Electron and other web technologies, it's cross-platform compatible.
Step 2: Install Xcode Command Line Tools
Xcode is Apple's software developer kit for creating apps on Apple devices (iPhones, iWatches, Macs, etc.) It contains a bunch of stuff you probably don't want, but a few things you need for your Python setup, including the GNU Compiler Collection (aka, GCC).
The subset of Xcode that's useful to us—the Xcode Command Line Tools—can be installed with this command:
You'll see a prompt like this:
Click 'Install' to continue. Note that this step will take several minutes, so please be patient.1
Step 3: Install Homebrew
Homebrew is the de facto (though, unofficial) package manager for Mac. It allows you to install software that isn't pre-installed on your Mac and isn't available in the App Store.
Here's the command to install homebrew:
The output should look like this:
As prompted, press RETURN. Once again, you'll be asked to type in your password.
PLOT TWIST: You had Python all along (but don't use it)
Python has come pre-installed on every Mac since 2012. You can check which version you have installed with the following command:
The output will look like Python 2.7.15. The numbers after the second dot will vary.
So why are we jumping through all these hoops?
Because this 'system' Python is tightly integrated with your Mac's operating system, and it's maintained by Apple. Any one of those system updates that occasionally pop up in your App Store could modify this installation of Python or other software that relies on it. You don't want to disrupt these automatic updates. You also don't want to write any code that relies on this outdated Python installation that's not under your control.
Instead, you need to install a newer version of Python in a different location that's isolated from your system Python but still discoverable by your command line.
We could use Homebrew to install another version of Python, and a lot of other guides will suggest this route. However, the best tool we've found for tackling this problem is pyenv.
With pyenv, you can:
- Install any exact version of Python
- Install multiple versions of Python, all in isolation from each other; and
- Switch between Python versions without any fuss.
Also, pyenv has a handy plugin to help you manage virtual environments for your Python projects, which we will discuss later in this guide.
Go deeper
You can find out more about pyenv, how it works and how to use it in Managing Multiple Python Versions With pyenv from RealPython.com.
Step 4: Install pyenv's recommended build dependencies
pyenv installs Python by downloading and compiling it from source. The software we installed during Step 2 is critical to this process, but pyenv's documentation also recommends installing additional build dependencies when working on a Mac.
To satisfy these recommendations, type in brew's install command followed by the list of recommended software packages:
Step 5: Install pyenv
We recommend installing pyenv via the automatic installer provided by the project's maintainers:
This will install pyenv and several of its most useful plugins.
Step 6: Initialize pyenv and pyenv-virtualenv in your shell
At the bottom of the pyenv-installer's output, you should see something like this:
This message is letting you know that, while pyenv finished installing, the commands are not yet available to use. That's because your shell—the program you interact with via Terminal that actually executes the commands you type in—still needs help finding it.
Your shell environment includes a variable called PATH that contains a list of all the directories where the command line can find and execute commands. Type in the following command:
And you'll see the full list of directories, delimited by a :.
When, for example, you type python on the command line and press RETURN, your shell will read this list of directories from left to right, looking in each one for a program named 'python'. If it finds that program, the shell will execute that program. If not, it will respond with command not found.
So now you need to add pyenv's location to the front of this list of directories. That's what this line does:
Then you need to initialize pyenv:
And pyenv-virtualenv:
You could invoke these commands one at a time from the command line, but the results would only apply in your current session. Once you close your terminal window, you'll lose these changes.
Instead, you need to add these three lines to a shell initialization file that runs each time you start a new session in your terminal. Look back at the output from pyenv-installer, and you'll see the name of the file you need to modify, which is determined by which shell your terminal is running.
Most likely, you'll need to modify .bashrc, which is one of the config files for Bourne-Again Shell (aka, bash). Bash has been the default shell on Macs since the early 2000s, but starting with Catalina, which Apple is due to publicly release this month, macOS is switching to Z Shell (aka, zsh). If you've already upgraded, then you'll need to modify .zshrc.
Python 2.7 Mac Catalina
Either of these config files will be in your user account's home directory, which has the alias ~. Assuming you're on bash, you can open that file in a text editor:
You might see an error that looks like The file /Users/{your user name}/.bashrc does not exist. If this happens, double-check the name of the file mentioned in the warning printed by at the end of pyenv's installation script. It might recommend modifying ~/.bash_profile, ~/.profile, ~/.zshrc or the name of some other shell initialization file particular to your system's configuration.
If you're sure you have the right file name, but it doesn't exist, just create the file using the touch command:
If you still can't figure out which file to modify, check out the Unix shell initialization page in pyenv's wiki. It may also be time to reach out to your nearest sys admin.
Once you've opened the proper shell initialization file, copy these three lines:
Then paste them at bottom of the file, save your changes and close the text editor window.
Step 7: Restart your shell
The easiest way to do this is to close your current Terminal window, and open a new one.
You can then check if pyenv is working properly like so:
The output should look like this:
Step 8: Install the latest (stable) version of Python
Still with us? Now you can install whichever version of Python you want. Let's start by seeing what's available by passing the --list option to pyenv's install command:
As you'll see, there are many versions of Python and a variety of distributions of these versions. We recommend installing the official latest stable version. That would be 3.7.4 as of the original date this guide was published.
This step will take a while since pyenv builds Python from source with each installation.
You may notice that, as time passes, pyenv's list of Python versions available to install will become stale. You can update this list at any time using the update command, which is provided by another plugin in the pyenv-installer bundle:
Step 9: Switch your global Python
Your current 'global' Python installation is still system, as the pyenv global command will tell you:
Again, you don't want to muck around with the system's Python installation. So let's switch to the new version of Python we just installed:
You can also check which versions of Python you've previously installed by invoking pyenv's versions command:
Step 10: Set up a new Python project
By this point, you've fully switched over to the latest version of Python. We're now in the homestretch!
Just so you know: You likely won't ever need to repeat Steps 1 through 9, not until you buy a new computer at least.
In this final step, you'll set up a new Python project. This is a process you will repeat many times. After all, you're a prolific and unstoppable coder/journalist, and we expect you to keep on crushing it.
Each of your Python projects needs two things.
First, a Python project needs a directory (or folder) to hold all of your code and related assets. We suggest keeping each project directory inside another directory that's easy to find. For example, you might make a directory called 'Devel' (short for development) inside your home directory:
Now create a new project directory inside your directory of projects. Call it 'new-project':
Second, each Python project needs a virtual environment, which is just another directory that contains a Python installation, plus a bunch of other packages, all specific to that project. This allows you to keep the dependencies for each of your Python projects isolated from each other.
We can initialize a new virtual environment using the virtualenv-init command from the pyenv-virtualenv plugin:
Note that it's good practice to ensure your project's directory and virtual environment have the same name.
Then, we can connect our new Python project's directory to its virtual environment so that whenever we navigate into our project's directory, the virtual environment will be activated automatically.
First, invoke cd to change into your project's directory:
Then invoke the pyenv local command, passing in the virtual environment's name:
Notice how your command line's prompt changes to include the name of your project like (new-project). This is a visual indicator that the project's virtual environment is now active.
If you change back to your home directory:
The project's name will disappear from the prompt, indicating the virtual environment is now deactivated. Change back into the project's directory:
And the virtual environment is automatically reactivated. How convenient!
Now you can use pip, Python's standard package manager, to grab whatever third party packages your project needs.
For example, if this project were a web scraper, you would probably want:
- requests for handling HTTP requests; and
- bs4 (aka, Beautiful Soup) for parsing HTML content.
Additionally, we encourage new data journalists to work in Jupyter Notebooks because it allows you to mix code and prose together in the same document. You'll also appreciate the instant feedback that Jupyter provides for each code snippet you write, especially while you are still learning Python's syntax and the interfaces of the packages you import.
Go deeper...
Read Python Virtual Environments: A Primer from RealPython.org to learn more about the history behind virtual environments and the many tools available for managing them.
You did it!
Thanks for sticking with us through this journey. We hope you've found this guide helpful and informative. Now go forth and do great work!
An early version of this article directed users on macOS Mojave (10.14) to install additional Apple SDK headers. However, with changes introduced in more recent versions of the Xcode command line tools, this step is no longer necessary.
Senior EditorJames Gordon researched and developed new platforms for journalism, drawing on his background in coding, data analysis and product management.READ NEXT
Gimbals for video: B...
Tags
codingdatahow-toinnovationinnovation labjames gordonkat duncanpythonscrapingterminal
Share
RelatedStories
- News
- Series
- Events
- Newsletters
Python Os X Catalina
RJI Student Innovation CompetitionProjectsImpact with Context: Tool to Measure News ImpactJournalism Digital News Archive (JDNA)Potter Digital AmbassadorsTrusting NewsJournalism Institute
Administrative Offices, Suite 300
Python Download For Mac Catalina
Columbia, MO 65211573-884-9121 | Fax: 573-884-3824
rji@rjionline.org
Copyright © 2021 - Curators of the University of Missouri. All rights reserved.
DMCA and other copyright information. An equal opportunity/affirmative action institution. Privacy Policy. Terms of Service.