| Week | Content | |
|---|---|---|
| W01 | Contract Description, Introduction to Unix and Systems Programming (Chapter 1 - Unix System Overview (Stevens & Rago) Course Objectives, Course Contract, Literature, Anatomy of Unix OS, Key Elements for the Kernel, System Calls, Shell Programming, Unix History, Processors (CPU, GPU, APU, TPU, QPU) and UNIX, UNIX in the era of Cloud Computing (Datacenters, Hypervisors, VMs/VPSs/Containers, IaaS/PaaS/SaaS, Hadoop/HDFS), UNIX for AI (Tensorflow, python, Jupyter Notebooks and UNIX integration examples), UNIX for Blockchain (Hyperledger Fabric), Edge Computing (Latencies/Bandwidth/Unix Example), Green Computing (Technology Energy Consumption, Norway Example), Course Content |
01 syllabus |
| (Revision) of UNIX for Users/Programmers a) UNIX commands: passwd, man, whatis, apropos, info, help, users, who, whoami, finger. b) Revision of UNIX for Users/Programmers from EPL232: EPL232.20: Directory/File handling (mkdir, rmdir, cp, mv, rm), File Display and Conversion (file, head, tail, cat, more, wc, sort, uniq, iconv), In/Out Streams (stdin, stdout, stderr), Prepending Data to file (3 methods), File Redirection, Piping; EPL232.21: UNIX commands (echo, touch), Ownership and Access Rights(chmod, chgrp, chown, umask, suid, sgid, sticky bit), Access Control Lists (ACLs): getfacl, setfacl. Regular Expressions in BRE and ERE (grep, egrep), Usage and Examples. EPL232.22: Process Control (jobs, ps, fg, bg, kill, top), Find File and Program (which, whereis, find, exec, xargs), UNIX Commands (alias, cut, tr, tee, mail, comm, diff, crontab). |
02 | |
| W02 | Unix Administration I - File System, Ownership and Access Rights
File system structure, Navigation in file system structure (ls, cd, pwd, pathnames, pushd, popd), Linux Logs (var/log/syslog, auth.log, kern.log, cron, apache2, mariadb, mysql, systemd / journalctl), disk space and partitions (du, df, mkfs.ext4, fdisk, lsblk), compression-decompression (zip, gzip, bzip2, unzip, gunzip, bunzip2, tar), links (symbolic, hard - ln). Ownership and Access Rights (chmod, chgrp, chown, umask, suid, sgid, sticky bit), Groups and User Accounts (groups, id, newgrp, LDAP getent, Useradd and LDAP luseradd). |
03 |
| UNIX Administration II - Volumes/Partitions, Packages, Network, Keys, Data Scraping, Identifiers Booting/Partitioning/Resizing (parted,gparted,resize2fs,df -h), macOS APFS Filesystem (Volume, Partition, Container), Data vs. System Volume, Signed System Volume (SSV) in macOS Big Sur. Example csrutil to disable/enable System Integrity Protection, Package Installation (Debian, Redhat and MacOSX/macOS): grub, rpm, dpkg, wget, yum, apt-get, port, creating yum mirror site; Network and Security: ipTables, tcpdump, nmap, netstat, nslookup, ifconfig, time servers timedatectl / timesyncd, public/private RSA keys with ssh-keygen and ssh-add, openssl, Public/Private Keys & AES256 Symmetric Cryptography, protecting server with iptables, protecting ssh daemon (ufw/iptables,fail2ban,systemctl), scp/ftps/sftp/ftp over ssh, ssh tunelling, Web and HTTP: curl, wget, complete web crawl with/without cookies. Identifiers, Digests and Binary-to-Text: date, $$, $RANDOM, uuidgen, md5sum, uuencode/uudecode, base64. Structured Data and Persistency of Shell Pipes using SQLite3 (Here String, Here Document for Shell Linkage, Examples), Apache Configuration and Webserver Stress testing (webhook stress testing), .htaccess Files (pushing data outside firewalls) |
04 | |
| W03 | UNIX Administration III - Hadoop Architecture Assignment 1 Announced Introduction to Cloud Computing: Typical Datacenters, Cloud Stack and Buzzwords, Public / Private Clouds, Utility Computing, Killer Apps, Economic Model, Distributed System Basics: I/O Performance, Replication Strategies; The Hadoop Project (Core, HDFS, Map-Reduce, HBase, HIVE), The Hadoop Distributed File System (HDFS), HDFS vs. NFS (Network File System), HDFS Example Deployments (Yahoo, Facebook) |
05 |
| Unix System Utilities: Sed Stream editor used to perform basic text transformations on an input stream (a file or input from a pipeline). Supplementary Manual: + Sed Introduction Manual |
06
sed.pdf |
|
| W04 | Unix System Utilities: Awk AWK is an interpreted programming language designed for finding and matching patterns as well as for performing actions on input files or streams. More on Awk dynamic type checking and the UNIX locale command. Supplementary Manual: + Awk Introduction Manual. |
07
awk.pdf |
| W05 | Bash Shell (Bash Guide - Chapters 1,2,3 Garrels) Assignment 2 Announced Introduction to Bash Shell, Environment and Variables (Environment and Variables, Control Files Login / Logout, Shell Variables, Shell Commands) Shell Programming (Program Execution, User Input, Command-line Parameters) Supplementary Manuals: + Bash Guide for Beginners, Machtelt Garrels + Advanced Bash Shell Scripting Guide, Mendel Cooper |
08
beg-bash.pdf adv-bash.pdf |
| Bash Programming (Bash Guide - Chapters 7,9,10 Garrels) Control Conditions (If, Case), Logical Operators (&&, II, !), Relational Operators (-gt,-lt,..), Numerical Expressions and Arrays, Iterative and Nested Loops, File Testing, Functions, Arrays and Examples, Multiple I/O example, Distributed "ps -ef" with ssh, Multiline Append and Reverse (cat vs tac), Multi-line/Single-Line Addition (awk, bc, python, bash). |
09 | |
| W06 | Advanced Bash Shell Programming (Bash Guide - Chapter 12 and selected topics from Advanced Bash Programming) Debugging with set, Signal Handling (signals & trap), Examples Solution, Dialogs and Examples |
10 |
| Low-Level I/O in C - Chapter 3 (Stevens & Rago) Error management with |
11 | |
| W07 | Files & Directories - Chapter 4 Stevens & Rago Lecture 9 completion, File metadata (sys/stat.h): Structure and Access (atime, mtime, ctime, noatime, nomtime, mount, examples), File Management with System Calls [rename(), chmod(), link(), unlink(), symlink(), readlink()], Advanced I/O: Flushing Kernel Buffers (O_DIRECT) and Volatile Cache (fsync(), fdatasync()), Disabling ATIME updates at mount time and on file open (O_NOATIME), Directory Management (mkdir(), rmdir()), Directory Access with dirent.h library [opendir(), readdir() και closedir()], Usage Examples (mystat, filetype, lsdirR), Memory Mapped Files (mmap/munmap) |
12 |
| Process Environment & Control - Chapters 7,8 Stevens & Rago Processes: Introductory Concepts, Process Identity (getpid(), getppid(), getuid(), getgid()), Process in Memory (Structure & Representation), Process creation (fork()), Environment Variables, Orphan Processes, Processes and Files, Process Waiting (wait(), waitpid()), Zombie Processes |
13 | |
| W08 | Processes and Signals - Chapters 9,10 Stevens & Rago
Lecture 11 completion, Function family exec(), system call system(), Signals in C, system call signal(), Signals and Processes, system call alarm(), Inter-process Signals (kill()). |
14 |
| Expected MIDTERM Exam: Check Lecture 1 for exact date. You will be permitted to have an A4 double-side cheating sheet for the exam (any font size) that contains book or slide material (contigent upon instructor proofing). |
||
| W09 | Inter-Process Communication: i)Pipes και ii)Named Pipes (FIFO) - Chapter 15 Stevens & Rago Assignment 3 Announced Lecture 12 completion, System Calls: dup, dup2, Communicatoin between Processes - Introduction, Problem and Solution Overview, IPC1: Pipes: Child=>Parent, child<=>Parent communication, Pipe Shell Implementaton, IPC2: FIFO (Named Pipes), FIFO and Shell, FIFO in C, Example Reader-Writer with FIFO. |
15 |
| XSI IPC (System V) : iii) Message Queues, iv) Shared Memory and v) Semaphores - Chapter 15 Stevens & Rago Definitions (Identifiers and Keys, IPC Resource Management, IPC Management with shell commands. Message Queues (M.Q.): M.Q. Structure, Creation, Message Send/receive, Example, M.Q. Control. Shared Memory: Structure, Creation, Attach / Detach, Control, Example of 2 processes communicating through shared memory. Semaphores: Structure, Creation, Handling, control, Semaphore Server Example. |
16 | |
| W10 | XSI IPC & Socket Programming - Chapter 16 Socket Programming, Socket Definition, Ports, Port Number Assignment, Socket Families, Socket Types, TCP Client/Server Communication, Client/Server Basic Principles. |
17 |
| Socket Programming - Chapter 17 Stevens & Rago Byte sequence conversion, Socket Descriptors, Socket Creation (socket()), Socket Address Structure, Συσχέτιση Διεύθυνσης με Υποδοχή (bind()), Listening of Incoming Connections (listen()), Acceptance of Incoming Connection Requests (accept()), Connection Request (connect()), Data transfer (read(), write(), send(), recv()), Connection Shutdown (close(), shutdown()), IP Address handling (inet_addr(), inet_ntoa()), Additional Useful Functions (getpeername(), gethostname(), gethostbyname(), gethostbyaddr()), getsockname(), setsockopt(), bzero(), bcopy()), Client-Server Example. Supplementary Manual: + Beej's Guide to Network Programming Using Internet Sockets, Brian Hall |
18
sockets.pdf |
|
| W11 | Multithreaded Programming - Chapter 11 Stevens & Rago Introduction and Thread Representation, Thread Advantages, Thread Types, Multithreaded Models, |
19 |
| Multithreaded Programming - Chapters 11-12 Stevens & Rago Assignment 4 Announced Data Inconsistency, Thread Synchronization with Binary Semaphores, Deadlock Avoidance and Reader/Writer Locks, Linux Threading: Native POSIX Thread Library, Related Topics: Thread Pools, Thread-Safe Libraries, sysconf() function. |
20 | |
| W12 | Selected Topic A: Hadoop/MapReduce System Anatomy Introduction to "Big-Data" Analytics (Example Scenarios and Architectures), Map-Reduce Programming Model, Microsoft's Dryad Programming Model, Map-Reduce Counting Problem Map-Reduce Architecture, Hadoop JobTracker, Tasktrackers and data-nodes, Failure Management, Map-Reduce Optimizations, Combiners, Compression, In-Memory Shuffling, Speculative Execution, Programming Map-ReducE with Languages / PIG and in-the-cloud. |
21 |
| W13 | Student Presentations Note: Both source code and PPT presentation must be submitted to Moodle until the evening before the presentation. |
Go |
| Student Presentations Note: Both source code and PPT presentation must be submitted to Moodle until the evening before the presentation. |
Go | |
| FINAL EXAM: Date, Time, Room - check UCY program. You will be permitted to have an A4 double-side cheating sheet for the exam (any font size) that contains book or slide material (contigent upon instructor proofing): https://www.ucy.ac.cy/fmweb/el/tomeas-spoudon/undergraduate-office/15-gr-articles/bottom-menu/49-exam-schedule |
- |