Introduction

In this article, I will show 3 methods to install the new fastai2 library using pip install. The 3 methods described here can be used to similar python libraries published on both pypi repository and on GitHub. I voluntarily omitted the conda install option because I wanted to focus on the methods that can be reproduced in any free cloud environment such as Google Colab or Kaggle.

Note: In this article, the different methods described are generic and therefore can be applied to any python package. You only have to replace the name fastai2 by the name of the corresponding python package such as torch

1- Installing from pypi repository

Note: The first method is the most common way to install a stable version of a python package. It will be the recommeded one once fastai2 will be officially released.

Although, fastai2 is still under development and its official release is expected to be during the next summer, it is important to be familiar with this first method especially if you intend to use fastai2 in a production environment.

To install fastai2 from fastai2 pypi repository, type the following command either from:

  1. the terminal:

    pip install fastai2

  2. or from a Jupyter notebook cell:

    !pip install fastai2

The difference between the 2 options is the use of the ! in the case of Jupyter notebook. However, the result is the same, the fastai2 package will be installed in the current active virtual environment.

Important: Following the best practices, it is important to create and activate a virtual environment before installing any python package. Lacking to do so, the package will be installed in your current virtual environment. This means that most likely, it will be installed in the base virtual environment in Ananconda (base is the default virual environment created when you installed Anaconda). Check out the Appendices A and B, here below, on how to respectively install and create a virtual environment.

Tip: You can create several virtual environments on the same machine. The following is the output of the different virtual environments that I created on my local machine.

Running the following command:

conda env list

outputs the list of my current virtual environments:

# conda environments:
#
base                     /home/farid/anaconda3
fastai2                  /home/farid/anaconda3/envs/fastai2
fastai2_dev           *  /home/farid/anaconda3/envs/fastai2_dev
fastai                   /home/farid/anaconda3/envs/fastai
gluon                    /home/farid/anaconda3/envs/gluon
prophet                  /home/farid/anaconda3/envs/prophet

The asterisk (*) indicates the active virtual environment

To check where the fastai2 package has been install, run the following command:

pip show fastai2

This command displays the following useful information:

Name: fastai2
Version: 0.0.11
Summary: Version 2 of the fastai library
Home-page: https://github.com/fastai/fastai2
Author: Jeremy Howard, Sylvain Gugger, and contributors
Author-email: info@fast.ai
License: Apache Software License 2.0
Location: /home/farid/anaconda3/envs/fastai2/lib/python3.6/site-packages
Requires: scikit-learn, pandas, fastprogress, requests, scipy, pillow, torchvision, matplotlib, pyyaml, spacy, torch, fastcore
Required-by:

It displays the package name, the version number, the location where it was installed (with the name of the virtual environment: in our case it is fastai2), and some other information.

Note: At the present time (March 17th, 2020), the fastai version found on the pypi repository is 0.0.11

2- Installing from GitHub - Non-editable package

Note: This method is used by users who are interested in the latest and the greatest features found in the fastai2 Github repository. In general, those users are interested in using the latest fastai2 package without the intention to either contibute to the fastai2 project or build new modules that can be used as fastai2 extensions. If this sound a bit unclear, everything will be crystal clear once you have read the Method 3 section.

The second method consists in installing the fastai2 package from the fastai2 GitHub repository. This begs the question: Why would we be interested in doing so? The answer is simple: If a python package is under development, and changes in the source code are introduced on a regular basis, chances are the package found on the pypi repository is lagging behind. In order to use the latest fastai2 features you need to install fastai2 from its GitHub repository.

As illustration of a situation where this option is convenient, I will cite the example where Jeremy shared a Kaggle kernel in which he wanted to show how to use a new developed module called medical. The latter was under development, and was not available on the pypi repository. Therefore the solution was to install the latest fastai2 version (that includes the medical module) by running the following command either on:

  1. the terminal:

    pip install git+https://github.com/fastai/fastai2.git

  2. or from a Jupyter notebook cell:

    !pip install git+https://github.com/fastai/fastai2.git

By comaparing the command above with the one used in method 1, you might have noticed that we replaced fastai2 by git+https://github.com/fastai/fastai2.git: this means that we are directly installing the latest source code found in the fastai2 GitHub repository.

Important: When using this option, it is very important to also install the fastcore library using the very same method:

The command shown, here below, should be run after installing the fastai2 package from its repository in order to align the 2 versions:

either from terminal:

pip install git+https://github.com/fastai/fastcore.git

and from a Jupyter notebook cell:

!pip install git+https://github.com/fastai/fastcore.git

You may ask yourself why do we have to also install the fastcore library from GitHub given the fact that fastai2 will always install fastcore because it is part of its requirements. The answer relies on fact that fastai2 will install the latest version found in the pypi repository as opposed to the latest version found on GitHub. If the fastai2 GitHub version is using new features found in the fastcore latest GitHub version, it will lead to an incompatibility if we use fastai2 latest GitHub version combined with the fastcore pypi version: basically, if we had done so we would have been mixing 2 misaligned versions. Consequently, we will end up experiencing some issues, and some errors will ultimately surface. Therefore, it is better to follow the good practice by pip installing both of them using the method described here above.

3- Installing from GitHub- Editable package

Note: This method is used by developers who are usually either 1) actively contributing to fastai2 project by adding new features or fixing bugs, or 2) creating their own modules, and making sure that their source code stay in sync with the fastai2 latest version. Lacking to do so, they will ultimately realize that their code is incompatible with the fastai2 repository one.

The goal is to keep our library (that depends on fastai2) aligned to the fastai2 one hand, and to avoid to keep pip installing both fastai2 and fastcore every time a new version is released (this would have been the case had we used Method 2). Thankfully, there exists a magic method (Method 3) that consists in using a so-called an editable version of the target python package (i.e. fastai2 and fastcore)

Installing an editable version of fastai2 is pretty straightforward. All we have to do is to follow these 3 simple steps by running the following commands from a terminal console:

git clone https://github.com/fastai/fastai2
cd fastai2
pip install -e .

Basically, we are cloning the fastai2 repository, then hopping into the cloned fastai2 folder, and finally running the command to pip install the package (fastai2) from the current folder (hence the use of the dot .) as an editable version by using the -e switch.

We also have to follow the same steps for the fastcore library:

git clone https://github.com/fastai/fastcore
cd fastcore
pip install -e .

For those following along, and who are familiar with this option, they might notice there is a tiny discrepancy between this method and the one found on the fastai2 repository (shown here below):

git clone https://github.com/fastai/fastai2
cd fastai2
pip install -e ".[dev]"

The difference is the use of the [dev] part. Using the [dev] flag issue an order to the installer to pip install all the packages listed under the dev requirements (dev_requirements = nbdev). This means the nbdev package (its a fastai package) will be installed from the pypi repository. Given the fact that nbdev is also under development, we may experienced some issues caused by a misalignment between the 3 different fastai packages being fastai2, fastcore, and nbdev. For that reason, it is desirable to also install the nbdev package as an editable package as follow:

git clone https://github.com/fastai/nbdev
cd nbdev
pip install -e .

Now that we installed our editable package, how are we going to update it? The answer is very simple: we git pull our latest version from the corresponding GitHub repository. Therefore, for the fastai2 package, we just have to hop into its cloned forder and run the following command:

git pull

This command will both update our cloned version of the fastai2 repository and update our editable package had its version changed. We have to run the command, here above, for both fastai2 and fastcore.

Appendix A: How to install Anaconda

On linux 

$ curl -O https://repo.anaconda.com/archive/Anaconda3-2019.10-Linux-x86_64.sh

$ sha256sum Anaconda3-2019.10-Linux-x86_64.sh
46d762284d252e51cd58a8ca6c8adc9da2eadc82c342927b2f66ed011d1d8b53  Anaconda3-2019.10-Linux-x86_64.sh

$ bash Anaconda3-2019.10-Linux-x86_64.sh
$ source ~/.bashrc

Warning: Please replace the Anaconda3-2019.10-Linux-x86_64.sh by a more recent filename found in the [Anaconda installer archive] (https://repo.continuum.io/archive/) .

On Windows

Appendix B: Creating a conda enviroment

On either Linux and Windows run the following commands:

conda create -n fastai2 python=3.6 anaconda
conda activate fastai2

Tip: You can choose which version of python you would like to install. In the case here above, the version 3.6 will be installed.

Appendix C: Useful pip commands

  • Install a python package (e.g. fastai2) from the pypi repository:
pip install fastai2
  • Uninstall a python package (e.g. fastai2):
pip uninstall fastai2
  • Show some information about a package (e.g. fastai2)
pip show fastai2

This is what it displayed on my local machine, at the moment I published the present post:

Name: fastai2
Version: 0.0.11
Summary: Version 2 of the fastai library
Home-page: https://github.com/fastai/fastai2
Author: Jeremy Howard, Sylvain Gugger, and contributors
Author-email: info@fast.ai
License: Apache Software License 2.0
Location: /home/farid/anaconda3/envs/fastai2/lib/python3.6/site-packages
Requires: scikit-learn, pandas, fastprogress, requests, scipy, pillow, torchvision, matplotlib, pyyaml, spacy, torch, fastcore
Required-by:

Appendix D: Useful conda commands

  • Create a conda virtual environment called fastai2 with python 3.6 version
conda create -n fastai2 python=3.6 anaconda
  • Activate a conda virtual environment called fastai2 
conda activate fastai2
  • List all the conda virtual environments 
conda env list
  • Install a python package (e.g. torch) from the conda repository:
conda install torch -c torch

-c : indicate which channel to use to install the corresponding package

  • Uninstall a python package (e.g. torch):
conda uninstall torch
  • List all the packages installed in the current conda virtual environment
conda list
  • List all the packages installed that start with the prefix fast in the current conda virtual environment
conda list fast

Output on my machine indicating the package name, current installed version, and the channel:

# Name              Version                 Build  Channel
fastai2             0.0.12                  dev_0    <develop>
fastcache           1.1.0                   py36h7b6447c_0
fastcore            0.1.15                  dev_0    <develop>
fastprogress        0.2.2                   pypi_0    pypi
fastscript          0.1.4                   pypi_0    pypi

Note: You might have noticed that both my fastai2 and fastcore package share the same Build Channel name being dev_0: this indicates that they both are editable versions.

Conclusion

In this article, I presented an overview describing 3 methods to install any package published on both pypi repository and GitHub. I used fastai2 library as use-case. However, the methods described here can be applied to any other package: the name fastai2 can be replaced by the corresponding package name such as torch. I also explained the goal of each of these 3 methods, and which method to pick depending on your specific use-case.