Welcome to the Etherpad for the Software Carpentry workshop at the
University of Chicago!

Users are expected to follow the Software Carpentry code of conduct:
http://software-carpentry.org/conduct.html

All content is publicly available under the Creative Commons
Attribution License: https://creativecommons.org/licenses/by/4.0/

Software Carpentry workshop
Research Computing Center
University of Chicago
July 11–12, 2017

Welcome to the workshop! 

During the workshop, we will use this Etherpad for shared
note-taking—please add notes and expand or correct others' notes as we
go along. If you have questions, please toss them into the chat (lower
right) and we'll answer them at the first convenient break.

Please fill out the pre-workshop survey if you haven't already:
https://www.surveymonkey.com/r/swc_pre_workshop_v1?workshop_id=2017-07-11-uchicago

Please also have a look at
https://rcc-uchicago.github.io/2017-07-11-uchicago which has the
schedule for each day and links to the lesson notes.

And please make sure that you have followed all the setup instructions
for your laptop:
https://rcc-uchicago.github.io/2017-07-11-uchicago/index.html#setup

Introductions
Name / Department or Program / Operating System (Windows, Mac or Linux)
Hossein Pourreza / Research Computing Center / Mac
Peter Carbonetto / Human Genetics & Research Computing Center / Mac
Yuxing Peng / Research Computing Center / Mac
Jonathan Skone / Research Computing Center / Mac
Po-Yuan Tung / Department of Medicine (postdoc) / Mac
Charlie Lang / Genetics and Systems Biology (PhD student) / Mac
Harry Ryu / Chemistry PhD student, Theoretical/Computational Chemistry / MacOS
Jim Franke / GeoSci Grad Student, Climate Change Impacts Modeling / Mac 
Sam Baureis / Undergraduate Student, Department of Human Genomics / Windows 
Meng Ye / IME / Windows
Atif Khan / Postdoc -Department of Medicine / Windows
Tasha Drake / Chemistry / Windows
JJ Bhongsatiern / Medicine / Windows
Mei Wang / Statistics Department / Mac
Arun Khattri / Postdoc Medicine / Mac
Edwin Arauz Diaz / Postdoc-BMB
Weerapat Pittayakanchit / Grad Student, PSD
Younan Li / Grad Student / BSD

Lessons

Introduction

Note: a PDF of the slides have been posted on the website.

Other recommended resources:
Software Carpentry - http://software-carpentry.org/lessons
Data Carpentry
- http://www.datacarpentry.org/lessons
Practical Computing for Biologists
- http://practicalcomputing.org
Effective Computation in Physics
- http://physics.codes


The UNIX Shell (Tuesday morning)

Shell bridges kernel and user.

ls: list contents of folder

ls -F : distinguishes files from directories

ls --help : manual for ls

f forward
b back

cd change directory

cd .. : change directory to level up

cd /: change directory to root folder

cd ~: change directory to home folder

ls -a: lists hidden folders

tab key: auto completes commands

tab key twice: shows all options

ls -1: all files in 1 column

ls -l: long version of file data
ls -l = ll

ls -l -t: sorts files by time

ls -s: sort files by size 

ls -r : reversed list

ctrl + c: stops the du command
ctrl+c usually cancells most of the operations (including du) in Linux

history: lists commands used in session

rm: deletes file

rmdir: deletes empty folder

create file: touch filename

rm -r: deletes folder and its contents **dangerous**  (r represents recursive)

rm -f: deletes multiple files by force (no asking about deletion)

cp file1 file2: copy files

cp -r folder1 folder2: copies folder and all of its contents

mv name1 name2: moves files into directories or change file names  **can override files** 

wildcards *(any number of characters) ?(one character)

wc filename: wordcount (lines words characters) 

wc -l filename: shows the number of lines in the file

cat *.txt | wc -l : pass output of command into the input of next command

cat *.txt > all.txt: combines all of the txt files into a single txt file

sort: sort

sort -n: sort by numbers

head: reads the beginning of a file

head -n 5 file: reads the first 5 lines of the file

sort < result = cat result | sort : result is the input of sort

>> : append to file (wc -l *.txt >> result)

echo : prints a given argument

for one in ABC DEF; do echo $one; done

for starts a command; done closes the command

one: variable name

for file in *.txt; do echo $file; head -n 1 $file; done: prints out the filename and 1st line of the file 

for one in 300 350 400; do mkdir temperature$one; done: makes three new folders "temperature300, temperature350, temperature400"

for one in {1..10}; do echo $one; done: prints out a list of 1, 2, 3, 4,..., 10

for one in {01..10}; do echo $one; done: prints out 01, 02, 03,..,10

for file in 'cat task.list'; do wc -l $file; done

for one in `ls`;do if [ -d $one ] ; then echo $one;fi;done

` ` : executes command

" ": interpreted; variable replacement

'  ': literal; no changes made 

echo $@ : all the arguments

echo $#: number of arguments

echo $1: first argument

echo $2: second argument

grep : 

Programming in Python (Tuesday afternoon)

Make meaningful variables for clarity

lists/array (ex. persons=[value1,value2..]

Look for packages relevant to your project and customize using import keyword

print(data[0,0:10]) represents data on days 0-9 [0,10)

Python in a high-performance computing environment (Wednesday morning)

Notes go here.

Version Control with Git (Wednesday afternoon)

to do:
- Install git.
- Make sure you can run git on your computer (e.g., "git --help").
- Please create a Github account (if you don't have one already).

Version control / change tracking tools: 
- Tracked changes in Microsoft word
- Time Machine (for Mac)
- Google docs

Collaboration / syncing tools: 
- Dropbox
- UChicago Box

Sharing / publishing tools: 
- Zenodo
- Data Dryad
- Google docs
- Uchicago box

Git commands we have learned so far:

git config
git help
git help -a
git <command> --help
git init
git status
git commit
git commit -a
git add 
git log
git log --oneline
git log --stat
git diff
git diff --staged
git show
git remote
git push
git clone
git pull