0% completed

All LessonsFree Lessons (6)

Course Introduction

Introduction What is Bash?

Project 1: Analyzing the 'US News' University Ranking Data

Unirank: Data Preview (head, pipe and csvlook)Find the colleges in the ranklist (grep, pipe and wc)Finding the percent of colleges in the ranklist (wc, grep)Listing the Institutes from a given state (grep)Finding the number of Institutes from each state Finding the correlation between university tuition and ranks Project demonstration Project Summary Test your skills

Project 2: Facebook Data Mining

FB: Data Preview (csvstat, csvcut, head, cat)Finding the number of status entry (csvcut, sort, uniq)Find the most popular status entry (awk, sort, cat, csvcut, head)Finding the most popular entry (Bash function)Project demonstration Project Summary Test your skills

Project 3: Australian Cities Crime Statistics

AU Crime Stats: Introduction

Row and Columns Statistics (sed, wc, head, cat)

Finding the top most crime in the whole country (sort, tail, awk)

Finding the top most crime per city (csvcut, sort, tail, awk)

Finding the best city in Australia! (Bash Shell Script)

Prject demonstration

Project Summary

Test your skills

Project 4: Shakespearean-era plays and poems data mining

Plays and Poems: Data Preview (cat, cut, head and csvlook)Counting the Plays and Poems (head, sed, grep, wc)Finding plays/poems by each author (sed, sort, uniq, head)Finding the most frequent words by Shakespeare Project Summary

Bash Tutorials

Hello! Bash Bash variables and functions Bash command execution and passing arguments Bash meta characters Bash quotation basics Bash read and store user input Bash redirections Bash conditional statements Bash `loop` statements Bash arithmetic Bash arrays

REGEX Tutorials

Hello! RegEx Basic Regular Expressions (BRE)Extended Regular Expressions (ERE)REGEX character classes REGEX look arounds REGEX atomic groups `(?>)`How to use REGEX in Bash shell scripting?

AWK Tutorials

Hello! AWK AWK Built-in Variables AWK built-in functions AWK useful examples

SED, GREP and Find Tutorials

Hello! SED - Stream Editor SED substitutions SED and regular expressions SED delete, print and grouping Hello! GREP Hello! Find command

Beyond the Text Files! Enter into the Big Data Landscape - Concepts

HDFS - Hadoop Distributed File System Map Reduce YARN - Yet Another Resource Negotiator!Flume SQOOP Hive Pig Spark HBase Big Data file formats

Conclusion

Conclusion and References

Learn Data Science with Bash Shell/

...

/

Data Preview

Data Preview

We'll cover the following...

Assuming you have saved your downloaded file as tmpB2ZP9eAFP---Crime-incidents-data.xls (from the data.gov.au), the first thing you want to perform is to convert ...

Ask