Data Science and Artificial Intelligence

Différences entre les versions de « Deep Learning Tutorial 2019 installations »

De Data Science and Artificial Intelligence
Aller à la navigation Aller à la recherche
m
 
(42 versions intermédiaires par un autre utilisateur non affichées)
Ligne 3 : Ligne 3 :
 
<hr />
 
<hr />
 
<p font-size:120%">
 
<p font-size:120%">
Participants needs to complete the 3 following instructions before the tutorial kick off date:
+
Participants MUST complete the 3 following instructions before the tutorial kick off date:
 
* Installations
 
* Installations
 
* Download resources
 
* Download resources
Ligne 10 : Ligne 10 :
  
 
<div>
 
<div>
 
  
 
<h3> Installations </h3><hr>
 
<h3> Installations </h3><hr>
 
There are 2 options:
 
There are 2 options:
# Using personal computer
+
# Using personal computer (alternative)
# Using Google Colab (alternative)
+
# Using Google Colab (preferred choice)
 +
 
 +
<b>NB: </b><i>Colab is the preference for this tutorial (at least after Tutorial 2) due to the need of a powerful GPU.<br> Nevertheless, participants are encouraged to do the installations on their personal computers in the case of unavailability/downtime of Google Colab for all users at the same time.</i>
 
<br><br>
 
<br><br>
  
 
<h5><u>Using personal computer </u></h5>
 
<h5><u>Using personal computer </u></h5>
 
A. Download and install Anaconda
 
A. Download and install Anaconda
<div class="toccolours mw-collapsible" style="width:70%; margin-left:70px; overflow:auto;" >
+
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
<i>Follow the underlisted steps to install Anaconda distribution in Linux (Ubuntu):<i>
+
# Download and install Anaconda depending on your OS from https://www.anaconda.com/distribution/#download-section (choose Python 3.6 version)
# cd /tmp
 
# curl -O https://repo.anaconda.com/archive/Anaconda3-2019.03-Linux-x86_64.sh
 
# sha256sum Anaconda3-2019.03-Linux-x86_64.sh
 
# bash Anaconda3-2019.03-Linux-x86_64.sh
 
# source ~/.bashrc
 
# conda list
 
 
 
<br> <i>Follow the underlisted steps to install Anaconda distribution in Windows<i>:
 
# coming soon ...
 
 
</div></div>
 
</div></div>
  
 
B. Create virtual environment
 
B. Create virtual environment
<div class="toccolours mw-collapsible" style="width:70%; margin-left:70px; overflow:auto;" >
+
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
<i>We will use the name `dsai` for thi purpose (if you choose a different name, endeavour to be consistent</i>:
+
<i>We will use the name `dsai` for this purpose (if you choose a different name, endeavour to be consistent)</i>:
 
# conda create --name dsai
 
# conda create --name dsai
 
# source activate dsai
 
# source activate dsai
 
</div></div>
 
</div></div>
 +
  
 
C. Install packages
 
C. Install packages
<div class="toccolours mw-collapsible" style="width:70%; margin-left:70px; overflow:auto;" >
+
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
 
<i>Install PyTorch and Tensorflow packages. Depending on your system, you can install either the cpu or gpu version. Do not install both:</i>
 
<i>Install PyTorch and Tensorflow packages. Depending on your system, you can install either the cpu or gpu version. Do not install both:</i>
# conda install pytorch torchvision cpuonly -c pytorch # for cpu
+
# conda install pytorch torchvision cpuonly -c pytorch # for cpu; if you don't have gpu
# conda install -c aaronzs tensorflow-gpu=1.10
+
# conda install -c aaronzs tensorflow=1.10
 
 
 
Or;
 
Or;
 
# conda install pytorch torchvision cudatoolkit=9.2 -c pytorch # for gpu with cuda 9.2
 
# conda install pytorch torchvision cudatoolkit=9.2 -c pytorch # for gpu with cuda 9.2
Ligne 57 : Ligne 49 :
 
# pip install torchsummary
 
# pip install torchsummary
 
# conda install -c anaconda scipy==1.1.0
 
# conda install -c anaconda scipy==1.1.0
# conda install -c conda-forge opencv tqdm
+
# conda install -c conda-forge opencv tqdm keras
 +
# conda install -c anaconda pillow
 +
# conda install -c anaconda scikit-learn
 +
# conda install -c anaconda scikit-image
 +
# pip install comet_ml
  
 
<i>For editor, install either jupyter notebook or jupyter lab:</i>
 
<i>For editor, install either jupyter notebook or jupyter lab:</i>
# conda install -c anaconda jupyter      
+
* conda install -c anaconda jupyter  
# conda install -c conda-forge jupyterlab
+
Or;   
<i> You may need to have other editors like notebook, sublime text, vscode, spyder, etc., if you wish.<i>
+
* conda install -c conda-forge jupyterlab
 
 
 
</div></div>
 
</div></div>
 +
<br><br>
  
  
 +
<hr/>
  
  
 +
<p><font color=blue>For visualization: install any web browser (e.g.: Google chrome, Mozilla Firefox, Microsoft Edge, Apple Safari, etc.,)</font></p>
  
<h3> Downloads</h3>
 
  
<h3> Configurations </h3>
 
  
 +
<h3> Downloads</h3><hr>
 +
There are 3 resources to download:
 +
* Codes
 +
* Models
 +
* Dataset
 +
<hr/>
  
* <b><big>Installation guide</big> </b><hr>
+
A. Codes
Open a terminal and type and execute the following commands:
+
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
<div class="toccolours mw-collapsible" style="width:90%; overflow:auto;">
 
<div style="font-weight:bold;line-height:1.6;">Anaconda</div>
 
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
Follow the following steps to install Anaconda distribution in Ubuntu:
+
# wget --content-disposition "[link expired]"
# cd /tmp
+
# unzip dsai_dl_tutorial.zip
# curl -O https://repo.anaconda.com/archive/Anaconda3-2019.03-Linux-x86_64.sh
+
# cd dsai_dl_tutorial
# sha256sum Anaconda3-2019.03-Linux-x86_64.sh
 
# bash Anaconda3-2019.03-Linux-x86_64.sh
 
# source ~/.bashrc
 
# conda list
 
 
</div></div>
 
</div></div>
  
<div class="toccolours mw-collapsible mw-collapsed" style="width:90%; overflow:auto;">
+
B. Models
<div style="font-weight:bold;line-height:1.6;">Python 3 virtual environment</div>
+
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
Follow the following steps to install:
+
<i>Navigate to the model directory and run the download.sh bash file:</i>:
# conda create --name dsai python=3
+
# cd models
# conda activate dsai
+
# chmod +x download.sh
 +
# ./download.sh
 
</div></div>
 
</div></div>
  
<div class="toccolours mw-collapsible mw-collapsed" style="width:90%; overflow:auto;">
+
C. Dataset
<div style="font-weight:bold;line-height:1.6;">TensorFlow</div>
+
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
Follow the following steps to install:
+
<i>Navigate to the dataset directory and run the download.sh bash file:</i>:
# conda install -c aaronzs tensorflow-gpu=1.10
+
# cd ../datasets
# conda update conda
+
# chmod +x download.sh
 +
# ./download.sh
 
</div></div>
 
</div></div>
  
<div class="toccolours mw-collapsible mw-collapsed" style="width:90%; overflow:auto;">
+
D. Slides
<div style="font-weight:bold;line-height:1.6;">Jupyter Notebook</div>
+
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
Follow the following steps to install:
+
<i>Download the presentation slides:</i>:
# conda install -c anaconda jupyter
+
# wget --content-disposition "[link expired]"
 
</div></div>
 
</div></div>
  
<div class="toccolours mw-collapsible mw-collapsed" style="width:90%; overflow:auto;">
 
<div style="font-weight:bold;line-height:1.6;">Other Libraries</div>
 
<div class="mw-collapsible-content">
 
Follow the following steps to install:
 
# conda install -c anaconda opencv3
 
# conda install -c anaconda matplotlib
 
# conda install -c anaconda pillow
 
# conda install -c anaconda scikit-learn
 
# conda install -c anaconda scikit-image
 
</div></div>
 
  
  
<br><br>
+
 
* <b><big> Tutorial dataset</big></b><hr>
+
<h5><u>Using Google Colab (alternative)</u></h5>
<div class="toccolours mw-collapsible mw-collapsed" style="width:90%; overflow:auto;">
+
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
<div style="font-weight:bold;line-height:1.6;">Object Recognition</div>
 
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
Follow the following steps to install
+
<b>Colab is the preference for this tutorial (at least after Tutorial 2) due to the need of a powerful GPU</b>
</div></div>
+
*  Please make sure you have a working Gmail account with atleast 3GB of free space on your Google Drive.
 +
*  Ensure you have executed correctly the instructions: A, B & C above.
 +
*  Upload your '''dsai_dl_tutorial''' folder (''containing the codes, dataset and models'') to your Google drive.
 +
*  Note the path and update it on the notebook according to instructions provided by the instructor on the tutorial day.
  
<div class="toccolours mw-collapsible mw-collapsed" style="width:90%; overflow:auto;">
 
<div style="font-weight:bold;line-height:1.6;">Pose Estimation</div>
 
<div class="mw-collapsible-content">
 
Follow the following steps to install
 
 
</div></div>
 
</div></div>
  
  
<br><br>
 
* <b><big> Git repository</big></b><hr />
 
  
  
<br><br>
+
<h3> Configurations </h3><hr>
* <b><big> Reference materials</big></b><hr>
+
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
<div class="toccolours mw-collapsible mw-collapsed" style="width:90%; overflow:auto;">
 
<div style="font-weight:bold;line-height:1.6;">Lecture Notes and Slides</div>
 
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
Follow the following steps to install  
+
<i>Run check_packages.py to see any missing package. See the output and install any missing packages:</i>
</div></div>
+
* python check_packages.py
<div class="toccolours mw-collapsible mw-collapsed" style="width:90%; overflow:auto;">
+
 
<div style="font-weight:bold;line-height:1.6;">Reference papers and web resources</div>
+
 
<div class="mw-collapsible-content">
+
<i>Run either jupyter notebook or jupyter lab to view/run the tutorial notebooks</i>
Follow the following steps to install
+
* jupyter notebook
</div></div>
+
Or,
 +
* jupyter lab
 +
<i>If you completed up to this step, you are ready!</i>
 +
 
 +
Happy coding... see you soon!
  
  
<br /><br>
+
</div></div>
* <b><big> Survey </big></b><hr />
 
  
  
<br><br>
 
* <b><big> Exercises </big></b><hr>
 
  
 
<br><br>
 
<br><br>
 
</div>
 
</div>

Version actuelle datée du 30 septembre 2019 à 10:34

Instructions


Participants MUST complete the 3 following instructions before the tutorial kick off date:

  • Installations
  • Download resources
  • Configurations


Installations


There are 2 options:

  1. Using personal computer (alternative)
  2. Using Google Colab (preferred choice)

NB: Colab is the preference for this tutorial (at least after Tutorial 2) due to the need of a powerful GPU.
Nevertheless, participants are encouraged to do the installations on their personal computers in the case of unavailability/downtime of Google Colab for all users at the same time.


Using personal computer

A. Download and install Anaconda

  1. Download and install Anaconda depending on your OS from https://www.anaconda.com/distribution/#download-section (choose Python 3.6 version)

B. Create virtual environment

We will use the name `dsai` for this purpose (if you choose a different name, endeavour to be consistent):

  1. conda create --name dsai
  2. source activate dsai


C. Install packages

Install PyTorch and Tensorflow packages. Depending on your system, you can install either the cpu or gpu version. Do not install both:

  1. conda install pytorch torchvision cpuonly -c pytorch # for cpu; if you don't have gpu
  2. conda install -c aaronzs tensorflow=1.10

Or;

  1. conda install pytorch torchvision cudatoolkit=9.2 -c pytorch # for gpu with cuda 9.2
  2. conda install -c aaronzs tensorflow-gpu=1.10

Install python libraries:

  1. conda install numpy matplotlib
  2. pip install torchsummary
  3. conda install -c anaconda scipy==1.1.0
  4. conda install -c conda-forge opencv tqdm keras
  5. conda install -c anaconda pillow
  6. conda install -c anaconda scikit-learn
  7. conda install -c anaconda scikit-image
  8. pip install comet_ml

For editor, install either jupyter notebook or jupyter lab:

  • conda install -c anaconda jupyter

Or;

  • conda install -c conda-forge jupyterlab






For visualization: install any web browser (e.g.: Google chrome, Mozilla Firefox, Microsoft Edge, Apple Safari, etc.,)


Downloads


There are 3 resources to download:

  • Codes
  • Models
  • Dataset

A. Codes

  1. wget --content-disposition "[link expired]"
  2. unzip dsai_dl_tutorial.zip
  3. cd dsai_dl_tutorial

B. Models

Navigate to the model directory and run the download.sh bash file::

  1. cd models
  2. chmod +x download.sh
  3. ./download.sh

C. Dataset

Navigate to the dataset directory and run the download.sh bash file::

  1. cd ../datasets
  2. chmod +x download.sh
  3. ./download.sh

D. Slides

Download the presentation slides::

  1. wget --content-disposition "[link expired]"



Using Google Colab (alternative)

Colab is the preference for this tutorial (at least after Tutorial 2) due to the need of a powerful GPU

  • Please make sure you have a working Gmail account with atleast 3GB of free space on your Google Drive.
  • Ensure you have executed correctly the instructions: A, B & C above.
  • Upload your dsai_dl_tutorial folder (containing the codes, dataset and models) to your Google drive.
  • Note the path and update it on the notebook according to instructions provided by the instructor on the tutorial day.



Configurations


Run check_packages.py to see any missing package. See the output and install any missing packages:

  • python check_packages.py


Run either jupyter notebook or jupyter lab to view/run the tutorial notebooks

  • jupyter notebook

Or,

  • jupyter lab

If you completed up to this step, you are ready!

Happy coding... see you soon!