Data Science and Artificial Intelligence

Différences entre les versions de « Deep Learning Tutorial 2019 installations »

De Data Science and Artificial Intelligence
Aller à la navigation Aller à la recherche
m
 
(9 versions intermédiaires par un autre utilisateur non affichées)
Ligne 80 : Ligne 80 :
 
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
 
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
−
# wget --content-disposition "https://seafile.unistra.fr/f/edda24980da54200af52/?dl=1"
+
# wget --content-disposition "[link expired]"
 
# unzip dsai_dl_tutorial.zip
 
# unzip dsai_dl_tutorial.zip
 
# cd dsai_dl_tutorial
 
# cd dsai_dl_tutorial
Ligne 98 : Ligne 98 :
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
 
<i>Navigate to the dataset directory and run the download.sh bash file:</i>:
 
<i>Navigate to the dataset directory and run the download.sh bash file:</i>:
−
# cd ../dataset
+
# cd ../datasets
 
# chmod +x download.sh
 
# chmod +x download.sh
 
# ./download.sh
 
# ./download.sh
Ligne 106 : Ligne 106 :
 
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
 
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
−
<i>clone the git repository:</i>:
+
<i>Download the presentation slides:</i>:
−
# wget --content-disposition "https://seafile.unistra.fr/d/9e379d8b3a3949f6b2bb/"
+
# wget --content-disposition "[link expired]"
 
</div></div>
 
</div></div>
  
Ligne 117 : Ligne 117 :
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
 
<b>Colab is the preference for this tutorial (at least after Tutorial 2) due to the need of a powerful GPU</b>
 
<b>Colab is the preference for this tutorial (at least after Tutorial 2) due to the need of a powerful GPU</b>
−
 
 
*  Please make sure you have a working Gmail account with atleast 3GB of free space on your Google Drive.
 
*  Please make sure you have a working Gmail account with atleast 3GB of free space on your Google Drive.
−
*  Download this zip folder `link to be updated`, unzip it, and upload it to your Google drive.
+
*  Ensure you have executed correctly the instructions: A, B & C above.
−
*  Follow instructions A,B & C above to download the models and datasets.  
+
*  Upload your '''dsai_dl_tutorial''' folder (''containing the codes, dataset and models'') to your Google drive.
−
*  Unzip and upload both downloads to your Google drive.
 
 
*  Note the path and update it on the notebook according to instructions provided by the instructor on the tutorial day.
 
*  Note the path and update it on the notebook according to instructions provided by the instructor on the tutorial day.
  
Ligne 132 : Ligne 130 :
 
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
 
<div class="toccolours mw-collapsible" style="width:70%; margin-left:40px; overflow:auto;" >
 
<div class="mw-collapsible-content">
 
<div class="mw-collapsible-content">
−
<i>Ensure every installation is completed successfully and every resources downloaded to their appropriate directory:</i>:
+
<i>Run check_packages.py to see any missing package. See the output and install any missing packages:</i>
−
<i>Execute check_packages.sh to see any missing package:</i>
+
* python check_packages.py
−
# cd ..
 
−
# chmod +x check_packages.sh
 
−
# ./check_packages.sh
 
−
 
 
−
Or run:
 
−
 
 
−
# python check_packages.py
 
  
−
<i>See the output of the bash file to install missing packages.</i>
 
  
 
<i>Run either jupyter notebook or jupyter lab to view/run the tutorial notebooks</i>
 
<i>Run either jupyter notebook or jupyter lab to view/run the tutorial notebooks</i>
Ligne 150 : Ligne 140 :
 
<i>If you completed up to this step, you are ready!</i>
 
<i>If you completed up to this step, you are ready!</i>
  
−
<i> Check that your tutorial organisation is in this hierarchical order:</i><br><br>
+
Happy coding... see you soon!
−
├── 01_introduction_to_dl_concepts<br>
 
−
│   └── README.md<br>
 
−
├── 02_a_primer_on_python<br>
 
−
│   └── README.md<br>
 
−
├── 03_introduction_to_pytorch<br>
 
−
│   ├── p01_pytorch-tensors.ipynb                  =>  Introduction to PyTorch tensors + autograd.<br>
 
−
│   ├── p02_pytorch_dataloaders.ipynb              =>  Introduction to PyTorch dataloaders.<br>
 
−
│   ├── p03_pytorch_model_training.ipynb          =>  Training a state-of-the-art model in PyTorch.<br>
 
−
│   ├── p05_extra_pytorch_mnist_training.ipynb    =>  A bonus program to train a a simple CNN on the mnist dataset. <br>
 
−
│   ├── p05_extra_pytorch_mnist_inference_live.py  =>  Bonus program to use the model trained above for live inference on the webcam stream.<br>
 
−
│   └── utils<br>
 
−
│      ├── hicoDataset.py                        =>  Custom data-loader class for hico-Det dataset<br>
 
−
├── 04_introduction_to_tensorflow<br>
 
−
│   └── README.md<br>
 
−
├── 05_a_dl_application<br>
 
−
│   └── ckpt                                      =>  Training weight and summary repository<br>
 
−
│   └── model                                      =>  DL model classes<br>
 
−
│   └── output                                    =>  Collection of outputs<br>
 
−
│   └── inference*.ipynb<br>
 
−
│   └── finetune*.ipynb<br>
 
−
│   └── README.md<br>
 
−
├── 06_generating_images_with_gans<br>
 
−
│   ├── README.md<br>
 
−
│   ├── Part A - GAN_image_generation.ipynb        =>  Using GANs to generate images from noise.<br>
 
−
│   └── Part B - GAN_syle_transfer.ipynb          =>  Using GANs to transfer image style from one image to another.<br>
 
−
 
 
  
  

Version actuelle datée du 30 septembre 2019 à 10:34

Instructions

Participants MUST complete the 3 following instructions before the tutorial kick off date:

  • Installations
  • Download resources
  • Configurations

Installations

There are 2 options:

  1. Using personal computer (alternative)
  2. Using Google Colab (preferred choice)

NB: Colab is the preference for this tutorial (at least after Tutorial 2) due to the need of a powerful GPU.
Nevertheless, participants are encouraged to do the installations on their personal computers in the case of unavailability/downtime of Google Colab for all users at the same time.

Using personal computer

A. Download and install Anaconda

  1. Download and install Anaconda depending on your OS from https://www.anaconda.com/distribution/#download-section (choose Python 3.6 version)

B. Create virtual environment

We will use the name `dsai` for this purpose (if you choose a different name, endeavour to be consistent):

  1. conda create --name dsai
  2. source activate dsai


C. Install packages

Install PyTorch and Tensorflow packages. Depending on your system, you can install either the cpu or gpu version. Do not install both:

  1. conda install pytorch torchvision cpuonly -c pytorch # for cpu; if you don't have gpu
  2. conda install -c aaronzs tensorflow=1.10

Or;

  1. conda install pytorch torchvision cudatoolkit=9.2 -c pytorch # for gpu with cuda 9.2
  2. conda install -c aaronzs tensorflow-gpu=1.10

Install python libraries:

  1. conda install numpy matplotlib
  2. pip install torchsummary
  3. conda install -c anaconda scipy==1.1.0
  4. conda install -c conda-forge opencv tqdm keras
  5. conda install -c anaconda pillow
  6. conda install -c anaconda scikit-learn
  7. conda install -c anaconda scikit-image
  8. pip install comet_ml

For editor, install either jupyter notebook or jupyter lab:

  • conda install -c anaconda jupyter

Or;

  • conda install -c conda-forge jupyterlab





For visualization: install any web browser (e.g.: Google chrome, Mozilla Firefox, Microsoft Edge, Apple Safari, etc.,)


Downloads

There are 3 resources to download:

  • Codes
  • Models
  • Dataset

A. Codes

  1. wget --content-disposition "[link expired]"
  2. unzip dsai_dl_tutorial.zip
  3. cd dsai_dl_tutorial

B. Models

Navigate to the model directory and run the download.sh bash file::

  1. cd models
  2. chmod +x download.sh
  3. ./download.sh

C. Dataset

Navigate to the dataset directory and run the download.sh bash file::

  1. cd ../datasets
  2. chmod +x download.sh
  3. ./download.sh

D. Slides

Download the presentation slides::

  1. wget --content-disposition "[link expired]"



Using Google Colab (alternative)

Colab is the preference for this tutorial (at least after Tutorial 2) due to the need of a powerful GPU

  • Please make sure you have a working Gmail account with atleast 3GB of free space on your Google Drive.
  • Ensure you have executed correctly the instructions: A, B & C above.
  • Upload your dsai_dl_tutorial folder (containing the codes, dataset and models) to your Google drive.
  • Note the path and update it on the notebook according to instructions provided by the instructor on the tutorial day.



Configurations

Run check_packages.py to see any missing package. See the output and install any missing packages:

  • python check_packages.py


Run either jupyter notebook or jupyter lab to view/run the tutorial notebooks

  • jupyter notebook

Or,

  • jupyter lab

If you completed up to this step, you are ready!

Happy coding... see you soon!





Récupérée de « https://dsai.icube.unistra.fr/index.php?title=Deep_Learning_Tutorial_2019_installations&oldid=224 »