Course Description: In this course, students will design and build a microprocessor-based embedded system application using a real-time operating system or RT POSIX extensions with Embedded Linux. The course focus is on the process as well as fundamentals of integrating microprocessor-based embedded system elements for digital command and control of typical embedded hardware systems. Standard project options include: stereo vision, computer vision, voice-over-IP, computer vision tracking systems, facial recognition and numerous related projects. The student will be introduced to the full embedded system lifecycle process in this course including: analysis, design (using extended finite state machine specification tools), programming, hardware assembly, unit testing, integration and system testing.
Lab Description: The course requires the student to install embedded Linux or an RTOS microkernel (FreeRTOS or Zephyr) on a System-on-Chip processor such as the Raspberry-Pi 3b+, NVIDIA Tegra K1 or Nano, the Altera DE1-SoC, or the Intel Curie / Quark (Arduino 101). In addition, students may want to set up an Oracle Virtual-Box Linux installation. For work with live video please note recommended cameras and make sure you have access to native JetPack Linux on a Jetson (TK1 or Nano). For other types of sensor interfaces and audio, the DE1-SoC or Curie/Quark is fine. This course must be completed using an embedded system RTOS (FreeRTOS or Zephyr) or embedded Linux only, not a PC running Linux. You will however find Linux as a useful host development system (Windows is ok too).
Week-1, Session-1 Class: Introduction - Lecture 1 Video , Lecture 1 Notes , Updated Notes - Course goals [10 weeks, 5 labs, Mid-Term Exam, 2 Quizzes] * Beagle xM and ARM DM architecture [ARM v7te ISA, Coretex A8] * Ubuntu Linux * Linux user space and kernel space programming * Hard Real-Time Theory * Drivers and Real-Time Video * 1/3 Theory, 1/3 Practice, 1/3 Practitioner Work - Course project [Final Extended Lab - USB Camera for Time Lapse Capture] - Course grading (see policy below) - Fall ECEE 5623 Video Files: Project from Previous Semesters (Robotics, Video, Audio) - RT Embedded System TERMINOLOGY INITIAL QUIZ REVIEW - Terminology Overview Linux Host Development System setup (Tips on Beagle xM or NVIDIA Jetson) - Booting Ubuntu and other Linux Distros on Beagle xM - Native host Linux (Ubuntu) or Oracle Virtual-Box Linux on Windows - See Nisheeth Bhat's IS on TimeSys for Hard Real-Time - Kaushik's Instructions for loading Ubuntu on Beagle xM - Beagle xM - "cat /proc/devices" - lsmod - dmesg - "tail -f /var/log/messages" - ECEE 5623 Example Linux Code Paper: Liu and Layland Rate Monotonic paper Read: RTECS (Real-Time Embedded Components and Systems) - Chapter 1 Start Lab-1 Week-1, Session-2 Class: Fundamentals of Real-Time Theory - Lecture 2 Video , Lecture 2 Notes , Updated Notes - What's makes distributed, real-time, embedded multitasking systems challenging? - Best effort, Hard real-time, Soft real-time - Scheduling Class Taxonomy - Best Effort scheduling (Round-Robin Timeslice Scheme - Review) - Intro to Fixed priority preemptive scheduling - Intro to Dynamic priority scheduling - Utility Curves Read: RTECS - Chapter 2 Paper: Lehoczky Sha and Ding paper Week-2, Session-1 Class: Fundamentals of RT Theory Cont'd - Services and High-Level Design - QUIZ on Terminology Overview, Lecture 3 Video , Lecture 3 Notes , Updated Notes INITIAL QUIZ - On Terminology - Complete on Canvas Real-Time Services - What a Service Release Timeline Really Looks Like - The CPU, I/O, Memory Resource Space (Characterizing RT Applications) - Intro to Timing diagrams (interference) - Intro to Hard real-time safe resource utilization bounds Case-Study #1: High-Level RT System Design Examples EXAMPLE - Space Infrared Telescope Facility Instrument Control (launched on August 25, 2003) - The hard real-time requirements and performance - What worked, what didn't and why - Overview Presentation - The Test Image EXAMPLE - DATA-CHASER Space Shuttle Hitch-hiker Class Payload Flown on STS-85, Summer 1997 Paper: Audsley DM paper INITIAL QUIZ - On Terminology - Complete on Canvas Lab-1 DUE Start Lab-2 Week-3, Session-1 Class: RM/DM Policy and Feasibility Test Derivation - Lecture 4 Video , Lecture 4 Notes , Updated Notes Rate Monotonic Policy and Feasibility Overview - Rate Monotonic Assumptions and Constraints - More on Fixed priority preemptive scheduling - Overview of Derivation of RM LUB - Hard real-time safe resource utilization bounds - More on Dynamic priority scheduling (Earliest Deadline First and Least Laxity First) - EDF and LLF Overview Introduction to Feasibility Tests - Sufficient Tests - RM LUB, DM - N&S Tests - Scheduling Point - Completion Test Deadline Monotonic Policy and Feasibility Overview - DM Sufficient Feasibility Test - DM Sufficient and More Necessary Test Beyond RM and DM: - Scheduling Point Feasibility Test - Completion Test Feasibility Test - Sufficient vs. Necessary and Sufficient Feasibility Tests - Computational Complexity of Feasibility Tests (O(1), O(n^2), O(n^3)) PROJECT BACKGROUNDER: COMPUTER VISION, Video-Lecture-Machine-Vision, PDF Download Read: RTECS - Chapter 3 Week-4, Session-1 Class: Services, Synchronization and Shared Resources - Lecture 5 Video , Lecture 5 Notes , Updated Notes Paper: PIP - "Priority inheritance protocols: an approach to real-time synchronization", Sha, L.; Rajkumar, R.; Lehoczky, J.P.; Computers, IEEE Transactionson, Volume: 39 Issue: 9, Sep 1990. HW and HW+FW Implementations of RT Services - ISRs, task canonical structure, and task/service release - FPGA/ASIC-based State Machines for Offloading - Microprocessor/Microcontroller Offloading - DSP Offloading SW Implementations of RT Services - Main+ISR Executive - Cooperative Non-Preemptive Threads - Callbacks - Continuation Functions - Software State-Machines - Priority Preemptive Run-to-Completion RTOS - Time-sliced Traditional Best Effort OS Synchronization and Resource Issues: - Problems with Blocking (resources other than CPU, e.g. I/O) - Break up into more threads (better scheduling control) - Interrupt driven I/O - e.g. Programmable FIFOs - Model Blocking Time - Priority inversion (general concept) - Unbounded priority inversion problem (mutex C.S.) - Priority inheritence - Priority ceiling Other Practical Problems - Solutions: 1) Estimating C (execution jitter) - use WCET 2) Period jitter - transform to highest frequency 3) ISR and context switch overhead - add to WCET 4) Deadlock/Livelock - HW Watch Dog 5) Bad code spinning and wedging - Sanity Monitoring Case-Study #2: Mars Pathfinder (As presented from multiple perspectives) Authoritative Mars Pathfinder Account Mike Jones Overview or What Happened to Mars Pathfinder Mars Pathfinder -- WRS Story Mars Pathfinder -- JPL Story PROJECT BACKGROUNDER: CONTINUOUS MEDIA (VIDEO), Video-Lecture-Media, PDF Download Read: RTECS -- Chapter 4, 5 & 6 Lab-2 DUE Start Lab-3 Week-5, Session-1 Class: Real-Time Service System Integration and IO (RT-Linux User-space and Kernel-space Service Threads and Synchronization) - Lecture 6 Video , Lecture 6 Notes , Updated Notes Service Synchronization, Communication, and IO - Using Message Queues to Sync Services and for Communication - Using Binary Semaphores to Sync Services - What about IO during Service Execution? Overlapping CPU and IO Cycles - Latency Hiding - Initial Release Input - Block DMA and FIFO Sensor Data - Intermediate IO - Memory-Mapped IO During Service Processing - Hiding IO latency with Overlap System Design with RT Services - Software process: analysis, design, specification, coding, unit testing, integration, system testing, delivery, operations - Implementation characteristics: tasks, message passing, synchronization, ISRs, memory-mapped I/O, bus interface, signals, shared memory, stack/data/heap mgt., comm. interfaces - Specifying the design of real-time software systems Data flow and Control flow (function and interfaces) Extended finite state machines and signal block diagrams MSC (multi-tasking multi-node protocol/comm model) - Processes, threads, and tasks, what's the diff in RT-Linux? Read: RTECS - Chapter 7 and re-read 4.2, 4.3, 4.4 regarding WCET Lab-3 DUE Week-6: EXAM-1 on RTECS Chapters 1-7 (2 hours to complete exam on Canvas) Week-6, Session-1 Class: Scalable Embedded Systems Architectures - Lecture 7 Video , Lecture 7 Notes , Updated Notes Intro to PCI & USB Architecture and I/O Architectures - Intro to PCI bus architecture and device interfaces - PCI Plug and Play Concept - USB Plug and Play Concept - Embedded System PCI Form Factors and Standards CompactPCI, PC/104+ (PCI+ISA), PMC, PCI-X 1.0a/b, PCI-X 2.0, PCI-Express Read: Universal Serial Bus, (chapters TBD) - Discussion on I/O Trends (High speed differential serial) USB PCI-Express 10G Ethernet (10GEA), which became IEEE P802.3ae Open Fabric Alliance for 10/20/40/80Gbps Infiniband Other High-speed Serial: Firewire, RapidIO, SAS/SATA - Discussion of ASIC Trends SoC (Yesterday's Board, Today's Chipset, Tomorrow's ASIC) Core + I/O (PowerPC 8xx, 82xx) Reconfigurable (Virtex II) Configurable (Tensilica) IP Modules (CPU Cores, Mem Ctlrs., Local Bus) Off-loading (Today's SW is tomorrow's HW) - The POSIX 1003.1b compliant Embedded system environment (VxWorks) - The POSIX Application Programmer's Interface - POSIX overview: message queues, RT signals, and semaphores - Clocks and timers Read: RTECS - Chapter 8 Start Lab-4 Week-7, Session-1 Class: Device Drivers and Characterization of Embedded I/O - Lecture 8 Video , Lecture 8 Notes , Updated Notes I/O interfaces - Digital - Analog (ADC, DAC interfaces) Microprocessor interface types (word or block) - Register-based control, status, data - Higher rate FIFO I/O - Block-oriented 1st/3rd party DMA tx/rx between I/O interfaces and memory - Bus burst transfers and block transfers - system memory map for MMIO devices - DRAM/SDRAM/DDR, BOOTROM, Flash External interface types - CPU local bus IO/MMIO E.g. PCI 2.x, GPIO, DRAM, Flash - Point-to-point or switched devices E.g. RS-232, RS-422, PCI-Express - Network multi-access devices E.g. Ethernet Device interfaces -- introduction to drivers - User space App interface (driver entry point interface to user processes) - Device interface (interface to devices) - ring buffers - blocking/non-blocking - ioctl "swiss army knife" - ISRs and signals/semaphores - scheduled I/O (handle buffering and processing in task) Read: RTECS - Chapter 9 Week-8, Session-1 Class: Real-Time Performance, Measuring Performance and HW/SW Debugging Techniques - Lecture 9 Video , Lecture 9 Notes , Updated Notes Mid-Term Solutions (Flipped Classroom) Estimating/Measuring Performance Based on CPU Architecture IBM Paper on Performance Monitoring and Tuning - Measuring/Controlling CPU Efficiency - Trace Ports (e.g. IBM PowerPC 4xx series, Strong Arm) - Built-in PMU (Performance Monitoring Units) (e.g. Intel Pentium, Xscale) - External Methods - Logic Analyzer Memory Traces (Cache Misses, DMA, Un-cached access) - Memory Port Markers (Writes to Un-cached Memory) - Profiling Code by Function or Block - Linux Software in Circuit Methods (e.g. Linux Ftrace, eLinux Ftrace, Kernel Shark, eLinux Kernel Shark) - VxWorks RTOS Software in Circuit Methods (e.g. Workbench Code Coverage using SW In-Circuit, Workbench Profiler) - Hardware Supported Profiling (e.g. Intel Vtune) - Cycle-based profiling - Event-based profiling - Cache Coherency - Harvard I-Cache, D-cache Architecture - Cache Invalidate, Flush, Lock, Pre-fetch - Measuring/Controlling I/O Efficiency - Bus Analyzers - e.g. PCI Event Traces - Logic Analyzer with Support Package - PCI Bus Performance Tuning - Bus Grant Arbiters and Priority Schemes, Minimum Burst Length - PCI-Express Performance Features - PCI Express Tutorial - Isochronal Virtual Channels - Split Transaction Paper: Zen of BDM Week-8, Session-2 Class: Embedded Volatile and Non-volatile Memory Devices - Lecture 10 Video , Lecture 10 Notes , Updated Notes Memory Technologies and Hierarchy - Harvard Arch. I-Cache, D-Cache (Separate instruction and data cache) - L1 On-Chip, L2/L3 Off-Chip Cache - SRAM (Fast, but expensive per bit stored) - DRAM, SDRAM, DDR (Double Data Rate DRAM), Quad Rate SRAM ECC (Error Correcting Circuitry) memory - Hardware/Software Design - Hamming encoding for EDAC (Error Detection and Correction) - review handout provided - ECC and maintenance and handling of SBEs and MBEs - Firmware ECC Req'ts: 1) initialize memory fully, 2) handle SBE interrupts, MBE check-stop, and 3) background scrubbing Non-Volatile Memory Devices and Systems - EPROM, EEPROM, NVRAM, and Flash (NAND, NOR) - Flash File Systems - TrueFFS - Solid-State-Disk (SSD) - Intel Tech Journal Article, also available on Papers Read in Class - Advanced Non-Volatile Memory/Storage Devices (PCM, FRAM, MRAM, Race-Track) - Overview Read: RTECS - Chapter 10 Lab-4 DUE Start Extended Lab and preparation of final submission Report and Canvas Digital Video Submission Week-9, Session-1 Class: High Availability vs. High Reliability (Same?) - Lecture 11 Video , Lecture 11 Notes , Updated Notes IBM developerWorks Paper on HA and HR Definition of High Availability (5 9's) - System is ready to provide service 99.999 % of any given service year - System can crash, but must reboot with minimal interruption to services and returning to fully service availability quickly Definition of High Reliability - Formal methods and testing to prove reliability is designed in and built in - SW Path coverage, statement coverage, and condition coverage criteria - HW Simulation testing with SW drivers - HW built-in test (Built-in Logic Analyzers, JTAG, Built-in Performance Monitoring Counters) - System Safe Modes Recovery Concepts - Conservative Safe Mode Manual Recovery (typical of satellite systems) - Automatic Recovery (typical of high availability sytems) Read: RTECS - Chapter 11, 12, 13, 14 Week-10, Session-1, EXAM-2 REVIEW, Extended Lab Q&A and In-Class Work Week-10: EXAM-2 - RTECS Chapters 8-14 (2.5 hours to complete exam on Canvas) Finals Week:EXTENDED LAB DUE ON Canvas DROP-BOX Grades Finalized per Registrar Calendar