Python Installation | Note of Thi

The last modifications of this post were around 1 year ago, some information may be outdated!

On this page

Windows
MacOS
Linux (Ubuntu)
Jupyer Notebook
Check GPU
pip
1. Update pip
2. Install packages with pip
pip vs conda?
Python virtual environnement
pyenv
Conda
Error?

Windows

✅ Update 11/Sep/20: Install python on WSL2 using Miniconda.

Other options (directly install)

Download and install Anaconda or smaller version Miniconda.

# default installed dir
C:\ProgramData\Anaconda3

Add to System Environment Variables:

C:\ProgramData\Anaconda3
C:\ProgramData\Anaconda3\Scripts

If you don't wanna use Anaconda and install it by yourself, add this:

C:\Users\<user>\AppData\Roaming\Python\Python36\Scripts

(You can find C:\...\Roaming by typing %appdata% in the Windows Explorer's navigation bar)

App to run: cmder (use this setting file).

MacOS

If you use Apple M1 (updated on 21/02/2021)?

AI things are limited! Pytorch is not available for M1 natively (If you wanna use it with Rosetta's help, read this article)!

💡 If you wanna install python2 environment, don't use conda create ..., use pyenv instead! Read this section for more.

👉 Instructions to install TensorFlow in a Conda Environment

conda init # after installing
# restart terminal after installing
# check if ok
which python # should return "/Users/thi/miniforge3/bin/python"

# manually install package
conda install <pkg name>
conda install -y <pkg> # auto accept

If you use conda (miniforge3), use the instructions in previous link to install tensorflow (not about the link to "newest" version of tensorflow-macos). If you wanna use python on your mac (using venv), check below link.

👉 Mac-optimized TensorFlow and TensorFlow Addons

Remark: After installing, check the version of tensorflow-macos (not tensorflow)

pip show tensorflow-macos

Remark: If you wanna install some package, use conda install -y <pkg>, then check its version by pip show <pkg>.

👉 Using my personal requirement file.

conda install --file requirement_m1.txt

By default, Python 2 is already installed on MacOS, you can check this by

python --version
# to be sure, check if python3 is installed?
python3 --version

In case you wanna make Python 3 as default on MacOS (For using things in VSCode, for example).

# Make sure: unlink to conda
conda deactivate
# Check current version
python --version # something: Python 2.7.18
which python # should: /usr/local/bin/python

# Suppose python3 is in
# /Users/thi/miniforge3/bin/python
sudo ln -s -f /Users/thi/miniforge3/bin/python /usr/local/bin/python
# Type your password
# Refresh current terminal
touch ~/.zshrc
# or
touch ~/.bashrc
# Check again
python --version # something: Python 3.9.1

Linux (Ubuntu)

Python is already installed on Ubuntu. You would like to install Anaconda, download and install it.

Wanna install Miniconda instead? 👉 Download .sh file and install inside Linux environement (including WSL2).

Miniconda or Conda? (ref)

Note that Conda is the package manager (e.g. conda list displays all installed packages in the environment), whereas Anaconda and Miniconda are distributions.

Choose Anaconda if you:

Are new to conda or Python
Like the convenience of having Python and over 1500 scientific packages automatically installed at once
Have the time and disk space (a few minutes and 3 GB), and/or
Don’t want to install each of the packages you want to use individually.

Choose Miniconda if you:

Do not mind installing each of the packages you want to use individually.
Do not have time or disk space to install over 1500 packages at once, and/or
Just want fast access to Python and the conda commands, and wish to sort out the other programs later.

Add conda to $PATH

# ADD CONDA TO $PATH

nano ~/.profile
# find where conda is installed and then
export PATH=/home/<user>/anaconda3/bin:$PATH
source ~/.profile

# (if you use zsh)
nano ~/.zshrc
export PATH=/home/<user>/anaconda3/bin:$PATH
source ~/.zshrc

# check
which python
# should return: /home/<user>/anaconda3/bin/python

# check version
conda --version

Make a right version

alias python=python3
alias pip=pip3
# for ubuntu >=20.04
sudo apt install python-is-python3
# prevent Python 2 from being installed as a dependency of something
sudo apt-mark hold python2 python2-minimal python2.7 python2.7-minimal libpython2-stdlib libpython2.7-minimal libpython2.7-stdlib

Jupyer Notebook

👉 Note: Jupyter notebook.

If you use VSCode, you should use its Jupyter Notebook extension, it's quick, clean and very easy to use.

Anaconda contains JN in it, no need to install it again. cd to the folder you wanna work on and run

# RUN (after installing Anaconda)
python -m notebook

# If `ImportError: DLL load failed`
active base # active env "base" in anaconda
jupyter notebook

The -m option allows you to execute a module or package as a script^[ref].

# If `import _ssl`, `ImportError`
set CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1
python -m notebook

Check GPU

👉 Note: Pytorch.

# with pytorch
import torch
print('cuda is available? ', torch.cuda.is_available())
print('device_count: ', torch.cuda.device_count())
print('current device: ', torch.cuda.current_device())
print('device name: ', torch.cuda.get_device_name(0))

# with tensorflow
import tensorflow as tf
print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU')))

pip

Update pip

# Check pip version
pip -V

# update pip
easy_install -U pip

Problem?

If you meet AttributeError: 'NoneType' object has no attribute 'bytes' when updating pip, check the version and make sure that there is only 1 pip on your computer and then use easy_install -U pip (don't forget to activate)
If there is a problem with python -m pip install --upgrade pip, use easy_install!

Install packages with pip

Install pip (It's actually installed with Anaconda). If you wanna upgrade it to the latest version:

python -m pip install --user --upgrade pip # install for current user only
python -m pip install --upgrade pip # need to run cmder as administrator

AttributeError: 'NoneType' object

First, activate <env> and then using easy_install -U pip. You can check the version of pip by pip -V.

# INSTALL A PACKAGE
pip install <package> # admin <-- SHOULDN'T!!!
pip install --user <package> # current user only

# LIST ALL INSTALLED PACKAGES
pip freeze

# REMOVE
pip uninstall <package>
pip uninstall --user <package>

# CHECK VERSION OF A PACKAGE
pip show <package>

# version <=
pip3 install -U "pillow<7"

# Install a package from a git repository
pip install git+https://github.com/TimeSynth/TimeSynth.git

If install packages with pip, they are installed in which environment of conda? Where pip is executed from.

which python
which pip

conda info --envs
# or
# conda env list

/c/ProgramData/Anaconda3/python
/c/ProgramData/Anaconda3/Scripts/pip

# conda environments:
base                  *  C:\ProgramData\Anaconda3
fastai                   C:\Users\thi\.conda\envs\fastai

Install packages with requirement file,

pip install -r requirements.txt

An example of requirement file

geopandas==0.4.1
grpcio==1.27.1
grpcio-tools==1.27.1
h5py==2.10.0
isodate==0.6.0
PyYAML==5.3.1

`pip` vs `conda`?

👌 Fact: each conda environment has a pip, just install package in each env with the corresponding pip.

conda activate base
which pip # /Users/thi/miniforge3/bin/pip

conda activate working
which pip # /Users/thi/miniforge3/envs/working/bin/pip

Old comments

Differences:^[ref]

pip installs python packages in any environment.
conda installs any package in conda environments.

Which one to be used?^[ref]

If you installed Python using Anaconda or Miniconda, then use conda to install Python packages. If conda tells you the package you want doesn't exist, then use pip (or try conda-forge, which has more packages available than the default conda channel).
If you installed Python any other way (from source, using pyenv, virtualenv, etc.), then use pip to install Python packages

Python virtual environnement

👌 Fact: For a simple life, use conda environment!

Click to show

👉🏻 Main guide is here.

sudo apt-get install python3-venv

# cd to <DIR> where python venv stored
python3 -m venv <DIR>

# activate
tutorial-env\Scripts\activate.bat # windows
source <DIR>/bin/activate # linux

# deactivate
deactivate

To detele, just remove the corresponding folder, i.e., <DIR>.

pyenv

👉🏻 Read more on SO. Note that, these methods didn't work on MacOS M1.

Conda

Install / Update conda

👉 Read more in this section.

# INSTALL CONDA BY PIP (without Anaconda)
pip install conda

# UPDATE CONDA
conda --version # check version
conda update -n base -c defaults conda

Error TypeError: LoadLibrary() argument 1 must be str, not None

Try to activate the environment base before running above line.

activate base # on Windows
source activate base # on MacOS

Install packages with conda

👌 Fact: Activate an environment conda, use its pip to install any package.

But you can use conda command to install package too

# INSTALL
activate <env> # you need to activate an environment first
conda install <package> # install for <env> only

# UPDATE
acctivate <env> # choose an env first
conda update <package> # ud package in that env

# LIST ALL INSTALLED PACKAGES
conda list

# check version of a package
pip show <pkg>

# Update packages listed in an env file to current env,
conda env update -n <env> -f /path/to/<file>.yml

# example of yml file
name: stats
dependencies:
    - python=3.6
    - geopandas==0.4.1

# install packages with requirements.txt
conda install --file requirements.txt

Conda's environements

👉 Check an official doc or this useful post.

Creating

# Create a new environment with python version 3.7
conda create -n <env-name> python=3.7 anaconda
# created in /home/thi/miniconda3/envs/<env-name>/
source activate <env-name> # activate this env

# The same python version with current shell's Python interpreter
conda create -n <env-name> python

# with addtional packages (python will be automatically installed)
conda create -n <env-name> <package1> <package2>
# with version
conda create -n <env-name> <package1>=1.16 <package2>

# in different directory
conda create --prefix /path/to/<env-name>

# create from file <file>.yml
conda env create -n <env> -f /path/to/<file>.yml

# Clone from another env
conda create --name <cloned-env> --clone <env>

Most of below commands are assumed to be run in an environment named env which is already activated. If you don't activate any environment before, use an alternative instead. For example,

conda update pandas             # <env> activated
conda update -n <env> pandas    # <env> isn't activated
conda update -p /path/to/<env>  # <env> isn't in the default directory of conda

conda env export -f <file>.yml                       # <env> activated & current folder
conda env export -n <env> -f /path/to/<file>.yml      # <env> isn't activated & different folder
conda update -p /path/to/<env> -f /path/to/<file>.yml # <env> isn't in the default directory of conda & different folder

# Activate an env
activate <env> # windows
source activate <env> # linux / macos

# DEACTIVE AN ENV
conda deactivate # Linux
deactivate # Windows
source deactivate # MacOS

# REMOVE AN ENV
conda remove -n <env> --all

# SHOW LIST OF CURRENT ENV
conda info --envs
# or
conda env list

# EXPORT env list TO A ENV FILE
conda env export -f <file>.yml

# Rename an env (conda >= 4.14)
conda rename -n old_name -d new_name

# Update packages listed in an env file to current env,
conda env update -n <env> -f /path/to/<file>.yml --prune

Show activated env in zsh

# run this first in your current shell
conda init zsh
# other options than zsh: bash, powershell, ...
source ~/.zshrc

If you wanna hide (base) in prompt,

conda config --set changeps1 false
# don't forget to reset the terminal

We cannot change the default environment in conda!

Kernel 2 & 3 for Jupyter Notebook

Check if nb_conda_kernels is installed by conda list. If not, install it by:

conda install nb_conda_kernels

If you are using Python 2 and you wanna separate Python 3,

conda create -n py37 python=3.7 ipykernel # "py37" is a custom name

If you are using Python 3 and you wanna separate Python 2,

conda create -n py27 python=2.7 ipykernel # "py37" is a custom name

Restart the Jupyter Notebook, the list of kernels is available under New.

Conda Revisions

# Check revisions
conda list --revisions

# Go back to revision `1`,
conda install --revision 1

Error?

# UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 975: ordinal not
# solution: instead of
pip3 install sesd
# use
LC_ALL=C.UTF-8 pip3 install sesd

# conda: The following packages are not available from current channels:
# Solution 1: One can use pip in this case (the same env with conda)
# Solution 2:
conda install -c anaconda <package>

# The following required packages can not be built: * freetype (from matplotlib)
# try to use conda to install matplotlib
conda install matplotlib
# it actually install the same thing as pip does on the same env

# dtaidistance: C-library is not available
pip install -vvv --upgrade --force-reinstall dtaidistance

# zsh: command not found: conda
# Make sure your installation folder is already
# in the $PATH
export PATH="/home/thi/miniconda3/bin:$PATH"