SkypeID: sam.siewert Google+ID: samuel.siewert Office: 928-777-6929 Cell: 303-641-3999 E-mail: email@example.com [E-mail: firstname.lastname@example.org]TA/Grader: Ashwin Iyengar, Ashwin.Iyengar@Colorado.EDU
Lab Description:The course requires the student to install Ubuntu Linux on a TI-OMAP Beagle xM or on an NVIDIA Jetson board. In addition, students may want to set up an Oracle Virtual-Box Linux installation. For work with live video please note recommended cameras and make sure you have access to native Ubuntu Linux. This course must be completed using an embedded system RTOS or embedded Linux only, not a PC.
Recommended platforms for the class: 1) Jetson TK1 from NVIDIA embedded using L4T, 2) Embedded Debian Linux on TI-OMAP Beagle xM or Beagle Bone Black (Debian likely pre-installed), 3) Altera DE1-SoC which can run Linux or FreeRTOS, 4) Native Linux Ubuntu LTS (12.04 or 14.04) download and install on any x86 laptop/desktop, 5) Oracle Virtual Box on Windows or Mac OS-X with Ubuntu LTS installation on VM (note that you can only process pre-recorded MPEG), 5) Use of MobaXterm or Xming/Putty and ECES server with ecee-gpu4 or ecee-gpu5. For Linux, once you have an Ubuntu platform set up, (excluding ECES server), you'll want to install OpenCV as described here and more generally here. If you have a Linux laptop, you can most likely use your built-in webcam (test this with class example code found here) or a simple UVC supported external webcam such as a Logitech C270. NOTE THAT YOU MUST COMPLETE YOUR EXTENDED LAB on an EMBEDDED Jetson, DE1-SoC or Beagle Board and that other Linux installations are intended to be used only for practice and support - i.e. you must use an embedded Linux or RTOS system to complete the course.
Week-1, Session-1 Class: Introduction - Lecture 1 , PDF Download - Course goals [10 weeks, 5 labs, Mid-Term Exam, 2 Quizzes] * Beagle xM and ARM DM architecture [ARM v7te ISA, Coretex A8] * Ubuntu Linux * Linux user space and kernel space programming * Hard Real-Time Theory * Drivers and Real-Time Video * 1/3 Theory, 1/3 Practice, 1/3 Practitioner Work - Course project [Final Extended Lab - USB Camera for Time Lapse Capture] - Course grading (see policy below) - Fall ECEN 5623 Video Files: Project from Previous Semesters (Robotics, Video, Audio) - RT Embedded System TERMINOLOGY INITIAL QUIZ REVIEW - Terminology Overview Linux Host Development System setup (Tips on Beagle xM or NVIDIA Jetson) - Booting Ubuntu and other Linux Distros on Beagle xM - Native host Linux (Ubuntu) or Oracle Virtual-Box Linux on Windows - See Nisheeth Bhat's IS on TimeSys for Hard Real-Time - Kaushik's Instructions for loading Ubuntu on Beagle xM - Beagle xM - "cat /proc/devices" - lsmod - dmesg - "tail -f /var/log/messages" - ECEN 5623 Example Linux Code Paper: Liu and Layland Rate Monotonic paper Read: RTECS (Real-Time Embedded Components and Systems) - Chapter 1 Start Lab-0 Week-1, Session-2 Class: Fundamentals of Real-Time Theory - Lecture 2, PDF Download - What's makes distributed, real-time, embedded multitasking systems challenging? - Best effort, Hard real-time, Soft real-time - Scheduling Class Taxonomy - Best Effort scheduling (Round-Robin Timeslice Scheme - Review) - Intro to Fixed priority preemptive scheduling - Intro to Dynamic priority scheduling - Utility Curves Read: RTECS - Chapter 2 Paper: Lehoczky Sha and Ding paper Week-2, Session-1 Class: Fundamentals of RT Theory Cont'd - Services and High-Level Design - QUIZ on Terminology Overview, Lecture 3 , PDF Download INITIAL QUIZ - On Terminology - Complete on D2L Real-Time Services - What a Service Release Timeline Really Looks Like - The CPU, I/O, Memory Resource Space (Characterizing RT Applications) - Intro to Timing diagrams (interference) - Intro to Hard real-time safe resource utilization bounds Case-Study #1: High-Level RT System Design Examples EXAMPLE - Space Infrared Telescope Facility Instrument Control (launched on August 25, 2003) - The hard real-time requirements and performance - What worked, what didn't and why - Overview Presentation - The Test Image EXAMPLE - DATA-CHASER Space Shuttle Hitch-hiker Class Payload Flown on STS-85, Summer 1997 Paper: Audsley DM paper INITIAL QUIZ - On Terminology - Complete on D2L Lab-0 DUE Week-3, Session-1 Class: RM/DM Policy and Feasibility Test Derivation - Lecture 4 , PDF Download Rate Monotonic Policy and Feasibility Overview - Rate Monotonic Assumptions and Constraints - More on Fixed priority preemptive scheduling - Overview of Derivation of RM LUB - Hard real-time safe resource utilization bounds - More on Dynamic priority scheduling (Earliest Deadline First and Least Laxity First) - EDF and LLF Overview Introduction to Feasibility Tests - Sufficient Tests - RM LUB, DM - N&S Tests - Scheduling Point - Completion Test Deadline Monotonic Policy and Feasibility Overview - DM Sufficient Feasibility Test - DM Sufficient and More Necessary Test Beyond RM and DM: - Scheduling Point Feasibility Test - Completion Test Feasibility Test - Sufficient vs. Necessary and Sufficient Feasibility Tests - Computational Complexity of Feasibility Tests (O(1), O(n^2), O(n^3)) PROJECT BACKGROUNDER: COMPUTER VISION, Lecture-Machine-Vision, PDF Download Read: RTECS - Chapter 3 Start Lab-1 Week-4, Session-1 Class: Services, Synchronization and Shared Resources - Lecture 5 , PDF Download "Priority inheritance protocols: an approach to real-time synchronization", Sha, L.; Rajkumar, R.; Lehoczky, J.P.; Computers, IEEE Transactionson, Volume: 39 Issue: 9, Sep 1990. Paper Available on:Paper's Read in Class Page HW and HW+FW Implementations of RT Services - ISRs, task canonical structure, and task/service release - FPGA/ASIC-based State Machines for Offloading - Microprocessor/Microcontroller Offloading - DSP Offloading SW Implementations of RT Services - Main+ISR Executive - Cooperative Non-Preemptive Threads - Callbacks - Continuation Functions - Software State-Machines - Priority Preemptive Run-to-Completion RTOS - Time-sliced Traditional Best Effort OS Synchronization and Resource Issues: - Problems with Blocking (resources other than CPU, e.g. I/O) - Break up into more threads (better scheduling control) - Interrupt driven I/O - e.g. Programmable FIFOs - Model Blocking Time - Priority inversion (general concept) - Unbounded priority inversion problem (mutex C.S.) - Priority inheritence - Priority ceiling Other Practical Problems - Solutions: 1) Estimating C (execution jitter) - use WCET 2) Period jitter - transform to highest frequency 3) ISR and context switch overhead - add to WCET 4) Deadlock/Livelock - HW Watch Dog 5) Bad code spinning and wedging - Sanity Monitoring Case-Study #2: Mars Pathfinder (As presented from multiple perspectives) Mike Jones Overview or What Happened to Mars Pathfinder Mars Pathfinder -- WRS Story Mars Pathfinder -- JPL Story PROJECT BACKGROUNDER: CONTINUOUS MEDIA (VIDEO), Lecture-Media, PDF Download Read: RTECS -- Chapter 4 & 5 Lab-1 DUE Start Lab-2 Week-5, Session-1 Class: Real-Time Service System Integration and IO (RT-Linux User-space and Kernel-space Service Threads and Synchronization) - Lecture 6 , PDF Download Service Synchronization, Communication, and IO - Using Message Queues to Sync Services and for Communication - Using Binary Semaphores to Sync Services - What about IO during Service Execution? Overlapping CPU and IO Cycles - Latency Hiding - Initial Release Input - Block DMA and FIFO Sensor Data - Intermediate IO - Memory-Mapped IO During Service Processing - Hiding IO latency with Overlap System Design with RT Services - Software process: analysis, design, specification, coding, unit testing, integration, system testing, delivery, operations - Implementation characteristics: tasks, message passing, synchronization, ISRs, memory-mapped I/O, bus interface, signals, shared memory, stack/data/heap mgt., comm. interfaces - Specifying the design of real-time software systems Data flow and Control flow (function and interfaces) Extended finite state machines and signal block diagrams MSC (multi-tasking multi-node protocol/comm model) - Processes, threads, and tasks, what's the diff in RT-Linux? Read: RTECS - Chapter 6 Week-6, Session-1 Class: Scalable Embedded Systems Architectures Lecture 7 , PDF Download Intro to PCI & USB Architecture and I/O Architectures - Intro to PCI bus architecture and device interfaces - PCI Plug and Play Concept - USB Plug and Play Concept - Embedded System PCI Form Factors and Standards CompactPCI, PC/104+ (PCI+ISA), PMC, PCI-X 1.0a/b, PCI-X 2.0, PCI-Express Read: Universal Serial Bus, (chapters TBD) - Discussion on I/O Trends (High speed differential serial) USB PCI-Express 10G Ethernet (10GEA), which became IEEE P802.3ae Open Fabric Alliance for 10/20/40/80Gbps Infiniband Other High-speed Serial: Firewire, RapidIO, SAS/SATA - Discussion of ASIC Trends SoC (Yesterday's Board, Today's Chipset, Tomorrow's ASIC) Core + I/O (PowerPC 8xx, 82xx) Reconfigurable (Virtex II) Configurable (Tensilica) IP Modules (CPU Cores, Mem Ctlrs., Local Bus) Off-loading (Today's SW is tomorrow's HW) - The POSIX 1003.1b compliant Embedded system environment (VxWorks) - The POSIX Application Programmer's Interface - POSIX overview: message queues, RT signals, and semaphores - Clocks and timers Lab-2 DUE Start Lab-3 Week-7, Session-1 Class: Device Drivers and Characterization of Embedded I/O Lecture 8 , PDF Download I/O interfaces - Digital - Analog (ADC, DAC interfaces) Microprocessor interface types (word or block) - Register-based control, status, data - Higher rate FIFO I/O - Block-oriented 1st/3rd party DMA tx/rx between I/O interfaces and memory - Bus burst transfers and block transfers - system memory map for MMIO devices - DRAM/SDRAM/DDR, BOOTROM, Flash External interface types - CPU local bus IO/MMIO E.g. PCI 2.x, GPIO, DRAM, Flash - Point-to-point or switched devices E.g. RS-232, RS-422, PCI-Express - Network multi-access devices E.g. Ethernet Device interfaces -- introduction to drivers - User space App interface (driver entry point interface to user processes) - Device interface (interface to devices) - ring buffers - blocking/non-blocking - ioctl "swiss army knife" - ISRs and signals/semaphores - scheduled I/O (handle buffering and processing in task) Week-7: MID-TERM, Mid-Term Review, PDF Download MIDTERM (2.5 hours to complete exam) Week-8, Session-1 Class (Mid-term solutions followed by lecture): Real-Time Performance, Measuring Performance and HW/SW Debugging Techniques Lecture 11 , PDF Download Mid-Term Solutions (Flipped Classroom) Estimating/Measuring Performance Based on CPU Architecture IBM Paper on Performance Monitoring and Tuning - Measuring/Controlling CPU Efficiency - Trace Ports (e.g. IBM PowerPC 4xx series, Strong Arm) - Built-in PMU (Performance Monitoring Units) (e.g. Intel Pentium, Xscale) - External Methods - Logic Analyzer Memory Traces (Cache Misses, DMA, Un-cached access) - Memory Port Markers (Writes to Un-cached Memory) - Profiling Code by Function or Block - Software in Circuit Methods (e.g. Workbench Code Coverage using SW In-Circuit, Workbench Profiler) - Hardware Supported Profiling (e.g. Intel Vtune) - Cycle-based profiling - Event-based profiling - Cache Coherency - Harvard I-Cache, D-cache Architecture - Cache Invalidate, Flush, Lock, Pre-fetch - Measuring/Controlling I/O Efficiency - Bus Analyzers - e.g. PCI Event Traces - Logic Analyzer with Support Package - PCI Bus Performance Tuning - Bus Grant Arbiters and Priority Schemes, Minimum Burst Length - PCI-Express Performance Features - PCI Express Tutorial - Isochronal Virtual Channels - Split Transaction Paper: Zen of BDM Week-8, Session-2 Class: Embedded Volatile and Non-volatile Memory Devices Lecture 12 , PDF Download Memory Technologies and Hierarchy - Harvard Arch. I-Cache, D-Cache (Separate instruction and data cache) - L1 On-Chip, L2/L3 Off-Chip Cache - SRAM (Fast, but expensive per bit stored) - DRAM, SDRAM, DDR (Double Data Rate DRAM), Quad Rate SRAM ECC (Error Correcting Circuitry) memory - Hardware/Software Design - Hamming encoding for EDAC (Error Detection and Correction) - review handout provided - ECC and maintenance and handling of SBEs and MBEs - Firmware ECC Req'ts: 1) initialize memory fully, 2) handle SBE interrupts, MBE check-stop, and 3) background scrubbing Non-Volatile Memory Devices and Systems - EPROM, EEPROM, NVRAM, and Flash (NAND, NOR) - Flash File Systems - TrueFFS - Solid-State-Disk (SSD) - Intel Tech Journal Article, also available on Papers Read in Class - Advanced Non-Volatile Memory/Storage Devices (PCM, FRAM, MRAM, Race-Track) - Overview Lab-3 DUE Start Extended Lab and preparation of final submission Report and D2L Digital Video Submission Week-9, Session-2 Class: High Availability vs. High Reliability (Same?) Lecture 13 , PDF Download IBM developerWorks Paper on HA and HR Definition of High Availability (5 9's) - System is ready to provide service 99.999 % of any given service year - System can crash, but must reboot with minimal interruption to services and returning to fully service availability quickly Definition of High Reliability - Formal methods and testing to prove reliability is designed in and built in - SW Path coverage, statement coverage, and condition coverage criteria - HW Simulation testing with SW drivers - HW built-in test (Built-in Logic Analyzers, JTAG, Built-in Performance Monitoring Counters) - System Safe Modes Recovery Concepts - Conservative Safe Mode Manual Recovery (typical of satellite systems) - Automatic Recovery (typical of high availability sytems) Week-10, Session-1, FINAL QUIZ REVIEW, Extended Lab Q&A and In-Class Work Final QUIZ Review: Lecture Final-Review , PDF Download Week-10 - Please take Final QUIZ on D2L 8/7/14, EXTENDED LAB DUE ON D2L DROP-BOX Grades Finalized per Registrar Calendar