ECEN 5623/4623 - Real-Time Embedded Systems, ESE Program

2015 Term-D Class: Wednesdays, Flipped Classroom using BlueJeans @ 8:00-9:00 PM Colorado [7:00PM Pacific/Arizona], Starting on June 3rd until August 5th, 2015

Join @ 8PM Colorado Time on Wednesdays - BlueJeans Meeting Link (Participants), Instructor Login Please watch Video Lectures and consult Lecture Notes before each flipped class. Please use one of the supported embedded platforms for the course along with supporting tools as outlined here.

Office Hours: Available by e-mail request.
SkypeID: sam.siewert
Google+ID: samuel.siewert
Office: 928-777-6929
Cell: 303-641-3999
E-mail: siewerts@erau.edu
[E-mail: siewerts@colorado.edu]
TA/Grader: Ashwin Iyengar, Ashwin.Iyengar@Colorado.EDU
NEW Office Hours: Tuesday and Wednesday 6:30-8:00 PM
[Week-1 Office Hours on 6/5 - 6:30-8:00 PM and on 6/6 - 3:00-4:30 PM]
Skype ID: ashwin.iyengar21


Remote Lab Note: Embedded Linux or FreeRTOS are BOTH acceptable platforms for this course and there are numerous options for your lab setup. Please review and purchase and configure the option that works best for you. Note that the NVIDIA Jetson can also be used for ECEN5653 and note that the DE1-SoC has an FPGA which may be used in other embedded courses.
Graduate Students with Questions - please e-mail beboulderanywhere@colorado.edu. Undergraduates please e-mail Jessica Drelles - Names A-K [Jessica.Drelles@Colorado.edu] or Beth Curtis - Names L-Z [Beth.Curtis@Colorado.edu]. For more general info on Be Boulder Anywhere e-mail Beth Morledge Webb [Elizabeth.Webb@Colorado.edu].
Please note that all course materials should be consulted on Desire-2-Learn; If for some reason you can't access D2L, this page is provided as a backup.


Course Description:In this course, students will design and build a microprocessor-based embedded system application using a real-time operating system or RT POSIX extensions with Embedded Linux. The course focus is on the process as well as fundamentals of integrating microprocessor-based embedded system elements for digital command and control of typical embedded hardware systems. Standard project options include: stereo vision, computer vision, voice-over-IP, computer vision tracking systems, facial recognition and numerous related projects. The student will be introduced to the full embedded system lifecycle process in this course including: analysis, design (using extended finite state machine specification tools), programming, hardware assembly, unit testing, integration and system testing.

Lab Description:The course requires the student to install Ubuntu Linux on a TI-OMAP Beagle xM or on an NVIDIA Jetson board. In addition, students may want to set up an Oracle Virtual-Box Linux installation. For work with live video please note recommended cameras and make sure you have access to native Ubuntu Linux. This course must be completed using an embedded system RTOS or embedded Linux only, not a PC.

Recommended platforms for the class: 1) Jetson TK1 from NVIDIA embedded using L4T, 2) Embedded Debian Linux on TI-OMAP Beagle xM or Beagle Bone Black (Debian likely pre-installed), 3) Altera DE1-SoC which can run Linux or FreeRTOS, 4) Native Linux Ubuntu LTS (12.04 or 14.04) download and install on any x86 laptop/desktop, 5) Oracle Virtual Box on Windows or Mac OS-X with Ubuntu LTS installation on VM (note that you can only process pre-recorded MPEG), 5) Use of MobaXterm or Xming/Putty and ECES server with ecee-gpu4 or ecee-gpu5. For Linux, once you have an Ubuntu platform set up, (excluding ECES server), you'll want to install OpenCV as described here and more generally here. If you have a Linux laptop, you can most likely use your built-in webcam (test this with class example code found here) or a simple UVC supported external webcam such as a Logitech C270. NOTE THAT YOU MUST COMPLETE YOUR EXTENDED LAB on an EMBEDDED Jetson, DE1-SoC or Beagle Board and that other Linux installations are intended to be used only for practice and support - i.e. you must use an embedded Linux or RTOS system to complete the course.


  1. "Real-Time Embedded Components and Systems", Sam Siewert, ISBN 1584504684, publisher link, Amazon link, Softpro Books, Barnes&Noble, also available in CU bookstore.
  2. Linux Kernel Development (3rd Edition), Robert Love, Addison-Wesley Professional; 3 edition (July 2, 2010), ISBN-10: 0672329468, ISBN-13: 978-0672329463, Linux Kernel Development (3rd Edition), by Robert Love

Important Course Links (Linux, RTOS=VxWorks, FreeRTOS is UNDER CONSTRUCTION)
Linux Example Code, RTOS Example Code, FreeRTOS Example Code
Linux Labs, RTOS Labs, FreeRTOS Labs
Linux Docs, RTOS Docs, FreeRTOS Docs, Linux Top-Errors, RTOS Top-Errors, 5623 Classic
Video Lectures
Papers Read in Class
RTECS 1st Ed CDROM
RTECS 2nd Ed Chapter Drafts
Course and Grading Policies General Policies

Important Course Tools
DE1-SoC Tools: DE1-SoC Board Terasic Page, ARM DS-5 Community Edition, Altera FTP Site, Altera DE1-SoC Labs, DE1-SoC Resources, Altera SoC EDS Downloads, Quartus II FPGA Design Tools,Class Documents for DE1-SoC.
NVIDIA Jetson Tools: NVIDIA Jetson Page, Jetson TK1 eLinux Page, Class Documents for Jetson.
TI Beagle Board Tools: Beagle Board Page, Class Documents for Beagle.
OpenCV
Oracle Virtual-Box
Ubuntu Linux LTS, 14.04 LTS Download
Linux UVC Supported Cameras, e.g. Logitech C200, or for Analog Camera Frame Grabber, Hauppauge USB-Live 2 used with any low-cost NTSC CCTV camera, e.g. Wired Color CCTV Security Camera - cost ranging from $10 up to $50.

Syllabus (See D2L for Definitive Lab Due Dates and Quizzes) Term-D - June 1 to August 7
Week-1, Session-1 Class: Introduction - Lecture 1 , PDF Download
        - Course goals [10 weeks, 5 labs, Mid-Term Exam, 2 Quizzes]
          * Beagle xM and ARM DM architecture [ARM v7te ISA, Coretex A8]
          * Ubuntu Linux
          * Linux user space and kernel space programming
          * Hard Real-Time Theory
          * Drivers and Real-Time Video
          * 1/3 Theory, 1/3 Practice, 1/3 Practitioner Work
        - Course project [Final Extended Lab - USB Camera for Time Lapse Capture]
        - Course grading (see policy below)
        - Fall ECEN 5623 Video Files: Project from Previous Semesters (Robotics, Video, Audio)
        - RT Embedded System TERMINOLOGY
          INITIAL QUIZ REVIEW - Terminology Overview

      Linux Host Development System setup (Tips on Beagle xM or NVIDIA Jetson)
        - Booting Ubuntu and other Linux Distros on Beagle xM
        - Native host Linux (Ubuntu) or Oracle Virtual-Box Linux on Windows
        - See Nisheeth Bhat's IS on TimeSys for Hard Real-Time
        - Kaushik's Instructions for loading Ubuntu on Beagle xM
        - Beagle xM
        - "cat /proc/devices"
        - lsmod
        - dmesg
        - "tail -f /var/log/messages"
        - ECEN 5623 Example Linux Code

       Paper: Liu and Layland Rate Monotonic paper

       Read: RTECS (Real-Time Embedded Components and Systems) - Chapter 1

Start Lab-0



Week-1, Session-2 Class: Fundamentals of Real-Time Theory - Lecture 2, PDF Download
        - What's makes distributed, real-time, embedded multitasking systems challenging?
        - Best effort, Hard real-time, Soft real-time
        - Scheduling Class Taxonomy
          - Best Effort scheduling (Round-Robin Timeslice Scheme - Review)
          - Intro to Fixed priority preemptive scheduling
          - Intro to Dynamic priority scheduling
        - Utility Curves

       Read: RTECS - Chapter 2

       Paper: Lehoczky Sha and Ding paper



Week-2, Session-1 Class: Fundamentals of RT Theory Cont'd - Services and High-Level Design - QUIZ on Terminology Overview, Lecture 3 , PDF Download

      INITIAL QUIZ - On Terminology - Complete on D2L

      Real-Time Services
        - What a Service Release Timeline Really Looks Like
        - The CPU, I/O, Memory Resource Space (Characterizing RT Applications)
        - Intro to Timing diagrams (interference)
        - Intro to Hard real-time safe resource utilization bounds

        Case-Study #1: High-Level RT System Design Examples

        EXAMPLE - Space Infrared Telescope Facility Instrument Control (launched on August 25, 2003)
        - The hard real-time requirements and performance
        - What worked, what didn't and why
        - Overview Presentation
        - The Test Image

        EXAMPLE - DATA-CHASER Space Shuttle Hitch-hiker Class Payload Flown on STS-85, Summer 1997

       Paper: Audsley DM paper


INITIAL QUIZ - On Terminology - Complete on D2L

Lab-0 DUE



Week-3, Session-1 Class: RM/DM Policy and Feasibility Test Derivation - Lecture 4 , PDF Download


      Rate Monotonic Policy and Feasibility Overview
        - Rate Monotonic Assumptions and Constraints
        - More on Fixed priority preemptive scheduling
        - Overview of Derivation of RM LUB - Hard real-time safe resource utilization bounds
        - More on Dynamic priority scheduling (Earliest Deadline First and Least Laxity First)
        - EDF and LLF Overview

      Introduction to Feasibility Tests
        - Sufficient Tests - RM LUB, DM
        - N&S Tests
          - Scheduling Point
          - Completion Test

      Deadline Monotonic Policy and Feasibility Overview
        - DM Sufficient Feasibility Test
        - DM Sufficient and More Necessary Test

      Beyond RM and DM:
        - Scheduling Point Feasibility Test
        - Completion Test Feasibility Test
        - Sufficient vs. Necessary and Sufficient Feasibility Tests
        - Computational Complexity of Feasibility Tests (O(1), O(n^2), O(n^3))


      PROJECT BACKGROUNDER: COMPUTER VISION, Lecture-Machine-Vision, PDF Download

      Read: RTECS - Chapter 3


Start Lab-1




Week-4, Session-1 Class: Services, Synchronization and Shared Resources - Lecture 5 , PDF Download

     "Priority inheritance protocols: an approach to real-time synchronization", Sha, L.; Rajkumar, R.; Lehoczky, J.P.; Computers, IEEE Transactionson, Volume: 39 Issue: 9, Sep 1990.

      Paper Available on:Paper's Read in Class Page

      HW and HW+FW Implementations of RT Services
        - ISRs, task canonical structure, and task/service release
        - FPGA/ASIC-based State Machines for Offloading
        - Microprocessor/Microcontroller Offloading
        - DSP Offloading

      SW Implementations of RT Services
        - Main+ISR Executive
        - Cooperative Non-Preemptive Threads
          - Callbacks
          - Continuation Functions
          - Software State-Machines
        - Priority Preemptive Run-to-Completion RTOS
        - Time-sliced Traditional Best Effort OS


        Synchronization and Resource Issues:
          - Problems with Blocking (resources other than CPU, e.g. I/O)
            - Break up into more threads (better scheduling control)
            - Interrupt driven I/O - e.g. Programmable FIFOs
            - Model Blocking Time
          - Priority inversion (general concept)
            - Unbounded priority inversion problem (mutex C.S.)
            - Priority inheritence
            - Priority ceiling

        Other Practical Problems - Solutions:
          1) Estimating C (execution jitter) - use WCET
          2) Period jitter - transform to highest frequency
          3) ISR and context switch overhead - add to WCET
          4) Deadlock/Livelock - HW Watch Dog
          5) Bad code spinning and wedging - Sanity Monitoring

      Case-Study #2: Mars Pathfinder (As presented from multiple perspectives)
        Mike Jones Overview or What Happened to Mars Pathfinder
        Mars Pathfinder -- WRS Story
        Mars Pathfinder -- JPL Story

       PROJECT BACKGROUNDER: CONTINUOUS MEDIA (VIDEO), Lecture-Media, PDF Download

      Read: RTECS -- Chapter 4 & 5

Lab-1 DUE

Start Lab-2



Week-5, Session-1 Class: Real-Time Service System Integration and IO
      (RT-Linux User-space and Kernel-space Service Threads and Synchronization) - Lecture 6 , PDF Download

      Service Synchronization, Communication, and IO
        - Using Message Queues to Sync Services and for Communication
        - Using Binary Semaphores to Sync Services
        - What about IO during Service Execution?

      Overlapping CPU and IO Cycles - Latency Hiding
        - Initial Release Input - Block DMA and FIFO Sensor Data
        - Intermediate IO
        - Memory-Mapped IO During Service Processing
        - Hiding IO latency with Overlap

      System Design with RT Services
        - Software process: analysis, design, specification, coding,
                           unit testing, integration, system testing,
                           delivery, operations

        - Implementation characteristics: tasks, message passing,
                                          synchronization, ISRs,
                                          memory-mapped I/O, bus
                                          interface, signals, shared
                                          memory, stack/data/heap
                                          mgt., comm. interfaces

        - Specifying the design of real-time software systems
          Data flow and Control flow (function and interfaces)
          Extended finite state machines and signal block diagrams
          MSC (multi-tasking multi-node protocol/comm model)


        - Processes, threads, and tasks, what's the diff in RT-Linux?

      Read: RTECS - Chapter 6




Week-6, Session-1 Class: Scalable Embedded Systems Architectures Lecture 7 , PDF Download

     Intro to PCI & USB Architecture and I/O Architectures
         - Intro to PCI bus architecture and device interfaces
         - PCI Plug and Play Concept
         - USB Plug and Play Concept
         - Embedded System PCI Form Factors and Standards
           CompactPCI, PC/104+ (PCI+ISA), PMC, PCI-X 1.0a/b, PCI-X 2.0, PCI-Express
         Read: Universal Serial Bus, (chapters TBD)


         - Discussion on I/O Trends (High speed differential serial)
           USB
           PCI-Express
           10G Ethernet (10GEA), which became IEEE P802.3ae
           Open Fabric Alliance for 10/20/40/80Gbps Infiniband
           Other High-speed Serial: Firewire, RapidIO, SAS/SATA

         - Discussion of ASIC Trends
           SoC (Yesterday's Board, Today's Chipset, Tomorrow's ASIC)
             Core + I/O (PowerPC 8xx, 82xx)
             Reconfigurable (Virtex II)
             Configurable (Tensilica)
             IP Modules (CPU Cores, Mem Ctlrs., Local Bus)
           Off-loading (Today's SW is tomorrow's HW)
        - The POSIX 1003.1b compliant Embedded system environment (VxWorks)
        - The POSIX Application Programmer's Interface
            - POSIX overview: message queues, RT signals, and semaphores
            - Clocks and timers

Lab-2 DUE

Start Lab-3


Week-7, Session-1 Class: Device Drivers and Characterization of Embedded I/O Lecture 8 , PDF Download

      I/O interfaces
        - Digital
        - Analog (ADC, DAC interfaces)

      Microprocessor interface types (word or block)
        - Register-based control, status, data
        - Higher rate FIFO I/O
        - Block-oriented 1st/3rd party DMA tx/rx between I/O interfaces and memory
        - Bus burst transfers and block transfers
        - system memory map for MMIO devices - DRAM/SDRAM/DDR, BOOTROM, Flash

      External interface types
        - CPU local bus IO/MMIO
            E.g. PCI 2.x, GPIO, DRAM, Flash
        - Point-to-point or switched devices
            E.g. RS-232, RS-422, PCI-Express
        - Network multi-access devices
            E.g. Ethernet

      Device interfaces -- introduction to drivers
        - User space App interface (driver entry point interface to user processes)
        - Device interface (interface to devices)
        - ring buffers
        - blocking/non-blocking
        - ioctl "swiss army knife"
        - ISRs and signals/semaphores
        - scheduled I/O (handle buffering and processing in task)


Week-7: MID-TERM, Mid-Term Review, PDF Download
      MIDTERM (2.5 hours to complete exam)



Week-8, Session-1 Class (Mid-term solutions followed by lecture): Real-Time Performance, Measuring Performance and HW/SW Debugging Techniques Lecture 11 , PDF Download
       Mid-Term Solutions (Flipped Classroom)


       Estimating/Measuring Performance Based on CPU Architecture

       IBM Paper on Performance Monitoring and Tuning

        - Measuring/Controlling CPU Efficiency
          - Trace Ports (e.g. IBM PowerPC 4xx series, Strong Arm)
          - Built-in PMU (Performance Monitoring Units) (e.g. Intel Pentium, Xscale)
          - External Methods
            - Logic Analyzer Memory Traces (Cache Misses, DMA, Un-cached access)
            - Memory Port Markers (Writes to Un-cached Memory)

          - Profiling Code by Function or Block
            - Software in Circuit Methods (e.g. Workbench Code Coverage using SW In-Circuit, Workbench Profiler)
            - Hardware Supported Profiling (e.g. Intel Vtune)
              - Cycle-based profiling
              - Event-based profiling

          - Cache Coherency
              - Harvard I-Cache, D-cache Architecture
              - Cache Invalidate, Flush, Lock, Pre-fetch

        - Measuring/Controlling I/O Efficiency
          - Bus Analyzers - e.g. PCI Event Traces
          - Logic Analyzer with Support Package

        - PCI Bus Performance Tuning
          - Bus Grant Arbiters and Priority Schemes, Minimum Burst Length
          - PCI-Express Performance Features - PCI Express Tutorial
            - Isochronal Virtual Channels
            - Split Transaction

        Paper: Zen of BDM




Week-8, Session-2 Class: Embedded Volatile and Non-volatile Memory Devices Lecture 12 , PDF Download

        Memory Technologies and Hierarchy
        - Harvard Arch. I-Cache, D-Cache (Separate instruction and data cache)
        - L1 On-Chip, L2/L3 Off-Chip Cache
        - SRAM (Fast, but expensive per bit stored)
        - DRAM, SDRAM, DDR (Double Data Rate DRAM), Quad Rate SRAM

        ECC (Error Correcting Circuitry) memory
        - Hardware/Software Design
        - Hamming encoding for EDAC (Error Detection and Correction) - review handout provided
        - ECC and maintenance and handling of SBEs and MBEs
        - Firmware ECC Req'ts:
          1) initialize memory fully,
          2) handle SBE interrupts, MBE check-stop, and
          3) background scrubbing

        Non-Volatile Memory Devices and Systems
        - EPROM, EEPROM, NVRAM, and Flash (NAND, NOR)
        - Flash File Systems - TrueFFS
        - Solid-State-Disk (SSD) - Intel Tech Journal Article, also available on Papers Read in Class
        - Advanced Non-Volatile Memory/Storage Devices (PCM, FRAM, MRAM, Race-Track) - Overview


Lab-3 DUE

Start Extended Lab and preparation of final submission Report and D2L Digital Video Submission


Week-9, Session-2 Class: High Availability vs. High Reliability (Same?) Lecture 13 , PDF Download

        IBM developerWorks Paper on HA and HR

        Definition of High Availability (5 9's)
        - System is ready to provide service 99.999 % of any given service year
        - System can crash, but must reboot with minimal interruption to services and returning to fully service availability quickly

        Definition of High Reliability
        - Formal methods and testing to prove reliability is designed in and built in
        - SW Path coverage, statement coverage, and condition coverage criteria
        - HW Simulation testing with SW drivers
        - HW built-in test (Built-in Logic Analyzers, JTAG, Built-in Performance Monitoring Counters)
        - System Safe Modes

        Recovery Concepts
        - Conservative Safe Mode Manual Recovery (typical of satellite systems)
        - Automatic Recovery (typical of high availability sytems)



Week-10, Session-1, FINAL QUIZ REVIEW, Extended Lab Q&A and In-Class Work
     Final QUIZ Review: Lecture Final-Review , PDF Download

Week-10 - Please take Final QUIZ on D2L

8/7/14, EXTENDED LAB DUE ON D2L DROP-BOX

Grades Finalized per Registrar Calendar