VxWorks 6.x ARM BSP Boot Process: A Step-by-Step Analysis

VxWorks 6.x ARM BSP Boot Process: A Step-by-Step Analysis

A Board Support Package (BSP) is the hardware-specific foundation that connects VxWorks to a target board. It initializes the processor and memory subsystem, configures board peripherals, establishes interrupt and timer infrastructure, and provides the device-driver interfaces required by the operating system.

When developing or porting a VxWorks BSP for an ARM platform, understanding the startup sequence is essential. The boot process is not simply a matter of jumping from ROM into the kernel. Instead, control progresses through several carefully ordered stages, beginning with low-level assembly initialization and eventually transitioning into the C-based kernel and application environment.

A typical VxWorks 6.x ARM startup sequence can be simplified as:

CPU Reset
   │
   ▼
romInit()
   │
   ├── CPU initialization
   ├── Memory controller setup
   ├── Stack initialization
   ├── Cache / interrupt preparation
   └── Determine boot type
   │
   ▼
romStart()
   │
   ├── Copy ROM image to RAM
   └── Decompress image if required
   │
   ▼
sysInit()
   │
   ├── Processor registers
   ├── Exception vectors
   ├── Interrupt masking
   └── Transition to C environment
   │
   ▼
usrInit()
   │
   ├── Cache initialization
   ├── BSS initialization
   ├── Exception setup
   ├── sysHwInit()
   └── kernelInit()
   │
   ▼
kernelInit()
   │
   ├── Kernel memory
   ├── Interrupt stack
   ├── TCB / root stack
   └── Spawn usrRoot
   │
   ▼
usrRoot
   │
   ├── Memory subsystem
   ├── Device initialization
   ├── sysHwInit2()
   ├── System clock
   └── Remaining OS services
   │
   ▼
VxWorks Runtime

Understanding this sequence also clarifies the responsibilities of the major BSP files and functions.

🧭 BSP Development Architecture

#

BSP development can generally be divided into five major stages.

Establish the development environment

#

Install and configure the appropriate VxWorks development environment, compiler, BSP tools, and target debugging infrastructure.

The exact environment depends on the VxWorks release. For VxWorks 6.x, Wind River Workbench and the corresponding toolchain are typically used.

Select a suitable BSP template

#

Starting from an existing BSP with a similar processor, evaluation board, memory controller, interrupt controller, or peripheral architecture can substantially reduce development effort.

The closer the reference BSP is to the target hardware, the more useful its startup assembly, memory initialization, interrupt handling, and device-driver implementations will be.

Implement pre-kernel initialization

#

The early boot stage executes before the VxWorks kernel exists.

Typical responsibilities include:

• CPU initialization
• Processor mode configuration
• Memory-controller initialization
• MMU configuration
• Cache configuration
• Initial stack setup
• Exception preparation
• Interrupt masking

This stage is primarily implemented in assembly because the C runtime environment has not yet been established.

Implement post-kernel hardware initialization

#

After the kernel starts, the BSP initializes the remaining board hardware and connects operating-system services to the hardware.

Typical responsibilities include:

• Interrupt-controller configuration
• System clock initialization
• Peripheral drivers
• Serial ports
• Network interfaces
• Storage devices
• Hardware-specific services

Test and verify the BSP

#

BSP correctness is fundamental to the stability of every software layer above it.

Testing should cover:

• Cold boot
• Warm reboot
• Memory initialization
• Interrupt handling
• System clock operation
• Cache and MMU behavior
• Device drivers
• Network interfaces
• Storage
• Long-duration operation
• Exception handling

🔧 Three Core BSP Functions

#

Although a BSP contains many files and functions, three functions are particularly important during initial development:

The startup sequence is therefore split between low-level assembly and higher-level C initialization.

🚀 Stage 1: Execution of romInit()

#

After reset, the ARM processor begins execution at the ROM entry point, typically _romInit() in romInit.s.

A simplified version looks like this:

_romInit:

Cold:

    MOV     r0, #BOOT_COLD

warm:

    B       start

    .ascii  "Copyright ..."

    .balign 4

start:

    /* Delay can avoid reset-related startup problems */

    TEQ     r0, #BOOT_COLD

    MOVEQ   r1, #INTEGRATOR_DELAY_VALUE

    MOVNE   r1, #1

delay_loop:

    SUBS    r1, r1, #1

    BNE     delay_loop

    ...

The r0 register carries the boot type into the startup code.

For a cold boot:

MOV r0, #BOOT_COLD

A warm boot uses a different entry path while preserving the required startup state.

Warm-Boot Detection

#

VxWorks can modify specific ROM instructions to distinguish a warm reboot from a normal reset.

For example, sysToMonitor() may inspect the first instructions of the ROM image:

if (p[0] == 0xE3A00002 && p[2] == 0x79706F43)
    pRom = (FUNCPTR)(ROM_TEXT_ADRS + 4);
else
    pRom = (FUNCPTR)ROM_TEXT_ADRS;

The important concept is that the warm-boot path can enter the ROM initialization code at a controlled offset rather than following the normal cold-boot entry point.

Responsibilities of romInit()

#

romInit() executes before the normal C environment exists, so it must be implemented in assembly.

Its responsibilities typically include:

• Initializing processor status
• Establishing an initial stack
• Configuring essential memory hardware
• Setting processor modes
• Configuring minimal MMU state where required
• Preparing caches
• Masking interrupts
• Determining cold versus warm boot
• Establishing enough hardware state to execute the next startup stage

The initial stack and memory environment established here are prerequisites for subsequent C execution.

Transition to romStart()

#

Once low-level initialization is complete, romInit() transfers control to romStart().

A typical ARM branch sequence is:

LDR     r12, L$_rStrtInRom
ORR     r12, r12, #1      /* Thumb mode */
BX      r12

The linker-generated constant identifies the romStart() address:

L$_HiPosn:
    .long ROM_TEXT_ADRS + HiPosn - FUNC(romInit)

L$_rStrtInRom:
    .long ROM_TEXT_ADRS + FUNC(romStart) - FUNC(romInit)

The boot type stored in r0 is passed into the next initialization stage.

For example, BOOT_COLD indicates that a cold boot path should be followed.

📦 Stage 2: Execution of romStart()

#

romStart() is responsible for moving the executable image from its ROM location into the runtime memory location.

This distinction is important because the image may initially execute from ROM or Flash, while the normal VxWorks runtime is intended to execute from RAM.

The simplified flow is:

ROM / Flash
    │
    │ copy
    ▼
RAM
    │
    ▼
sysInit()

If the image is compressed, decompression occurs as part of this transfer.

Copying the Image

#

For an uncompressed image, the copy operation may look like:

#ifdef UNCOMPRESSED

    ((FUNCPTR)ROM_OFFSET(copyLongs))
    (
        ROM_TEXT_ADRS,
        (UINT)K0_TO_K1(romInit),
        ROM_COPY_SIZE / sizeof(long)
    );

#else

    ((FUNCPTR)ROM_OFFSET(copyLongs))
    (
        ROM_TEXT_ADRS,
        (UINT)K0_TO_K1(romInit),
        ((UINT)wrs_kernel_data_end - (UINT)romInit) / sizeof(long)
    );

#endif

copyLongs() performs the actual memory transfer.

The exact addresses and copy lengths depend on the image layout generated by the BSP and build configuration.

After the image has been relocated into RAM, execution continues with the RAM-resident initialization code.

🧠 Stage 3: Execution of sysInit()

#

sysInit() is one of the first assembly routines executing from the relocated RAM image.

Its purpose is to establish the processor environment required before entering the higher-level C initialization code.

Typical responsibilities include:

• Initializing processor control registers
• Configuring exception-related state
• Disabling interrupts
• Establishing processor mode
• Configuring interrupt-controller access
• Preparing the transition into usrInit()

A representative section is:

MOV     r1, #0

MCR     CP_MMU, 0, r1, c13, c0, 0

/* Disable CPU and external interrupts */

MRS     r1, cpsr

BIC     r1, r1, #MASK_MODE

ORR     r1, r1, #MODE_SVC32 | I_BIT | F_BIT

MSR     cpsr, r1

MOV     r2, #IC_BASE

MVN     r1, #0

STR     r1, [r2, #FIQ_DISABLE-IC_BASE]

STR     r1, [r2, #IRQ_DISABLE-IC_BASE]

The key objective is to enter the C initialization phase with a controlled processor state and with asynchronous hardware activity suppressed.

⚙️ Stage 4: Execution of usrInit()

#

usrInit() is located in usrConfig.c and represents the first major C-level initialization stage of the VxWorks image.

At this point, enough processor and memory infrastructure exists for normal C code execution, but the complete multitasking kernel environment has not yet been established.

Hardware interrupts generally remain masked while usrInit() performs critical initialization.

Its major responsibilities include cache initialization, BSS clearing, exception setup, hardware initialization, and finally starting the kernel.

Initialize the Cache

#

The cache subsystem is initialized through:

cacheLibInit();

Depending on the BSP and configuration, instruction and data caches are enabled as the initialization sequence progresses.

Cache configuration must be consistent with the MMU and memory attributes established during the earlier boot stages.

Clear the BSS Segment

#

Uninitialized global and static variables occupy the BSS region.

The startup code clears this region:

bzero(edata, end - edata);

This establishes the C language requirement that zero-initialized global objects begin with zero values.

Initialize Exception Vectors

#

Exception vectors must be established before normal kernel operation and interrupt handling begin.

Typical calls include:

intVecBaseSet(...);
excVecInit(...);

intVecBaseSet() establishes the vector table location, while excVecInit() installs the standard exception handling infrastructure.

Initialize Board Hardware

#

The BSP’s first-stage hardware initialization is performed through:

sysHwInit();

The purpose of sysHwInit() is to place required hardware into the state expected by VxWorks.

This may include:

• Interrupt-controller initialization
• Peripheral reset/configuration
• Serial hardware
• Timer hardware
• Memory-mapped devices
• Board-specific control registers

The exact implementation is BSP-specific.

Start the Kernel

#

The final major responsibility of usrInit() is to invoke:

kernelInit();

This call transitions the system from single-threaded startup into the VxWorks multitasking environment.

A simplified parameter structure is:

typedef struct kernelInit_params
{
    FUNCPTR      rootRtn;
    unsigned int rootMemSize;
    char        *pMemPoolStart;
    char        *pMemPoolEnd;
    unsigned int intStackSize;
    int          lockOutLevel;
    unsigned int vmPageSize;
    unsigned int intStackOverflowSize;
    unsigned int intStackUnderflowSize;
    unsigned int idleTaskExcepStkSize;
    unsigned int idleTaskExcepStkOverflowSize;
    unsigned int idleTaskExcepStkUnderflowSize;
} _KERNEL_INIT_PARAMS;

The parameters define the initial kernel memory pool, root task, interrupt stack, virtual-memory page size, and related stack-protection settings.

🏃 Stage 5: Execution of kernelInit()

#

kernelInit() is provided by the VxWorks kernel rather than being normally implemented as part of the BSP.

Its purpose is to establish the core multitasking environment.

The initialization sequence includes operations such as:

1. Configuring the interrupt lock level.
2. Setting scheduler and round-robin parameters.
3. Allocating the interrupt stack.
4. Establishing kernel memory management.
5. Allocating the root task’s TCB.
6. Allocating the root task stack.
7. Creating the usrRoot task.
8. Transitioning the system into normal multitasking execution.

The important architectural boundary is that the system stops being merely a boot-time initialization flow and becomes a scheduler-controlled operating-system environment.

Once the root task is running, interrupts can be enabled and normal task scheduling begins.

🏠 Stage 6: Execution of usrRoot

#

usrRoot runs as a normal VxWorks task and performs the remaining system-level initialization.

At this point, the kernel’s multitasking environment is already active.

The initialization typically includes:

• Memory subsystem initialization
• I/O subsystem initialization
• Device-driver startup
• Filesystem initialization
• Network stack initialization
• System clock configuration
• Additional BSP-specific services

Initialize the Memory Subsystem

#

Depending on the VxWorks configuration, the system may initialize memory through functions such as:

memInit();

Other configurations may involve:

usrMmuInit();

or:

memShowInit();

The exact sequence depends on the VxWorks version and memory-management configuration.

Connect the System Clock

#

Once multitasking is active, the system clock can be connected:

sysClkConnect(...);

The clock initialization eventually invokes the BSP’s second-stage hardware initialization:

sysHwInit2();

This is an important distinction between sysHwInit() and sysHwInit2().

🔌 sysHwInit() vs sysHwInit2()

#

The two functions serve different phases of BSP initialization.

sysHwInit()

#

sysHwInit() executes during the early system initialization phase.

Its purpose is to configure the board into a safe and usable state before the normal multitasking environment is established.

Typical operations include:

• Initial peripheral setup
• Interrupt-controller initialization
• Device reset
• Board-specific hardware configuration
• Early serial-console initialization

Because this function runs early, it should avoid operations that depend on services not yet initialized.

sysHwInit2()

#

sysHwInit2() executes later, after the kernel and required system services are available.

It is therefore appropriate for operations such as:

intConnect(...);

and other interrupt-vector connections that depend on the VxWorks interrupt subsystem.

The conceptual sequence is:

sysHwInit()
    │
    ├── Early hardware configuration
    └── Safe hardware state
          │
          ▼
kernelInit()
          │
          ▼
usrRoot
          │
          ▼
sysHwInit2()
    │
    ├── Interrupt connections
    ├── Driver activation
    └── Late hardware initialization

This two-stage architecture prevents early boot code from depending on kernel services that do not yet exist.

⏱️ System Clock Initialization

#

The system clock is another important part of BSP startup.

After the clock driver is connected, the BSP configures the system clock rate through:

SYS_CLK_RATE

A common default configuration is approximately:

60 Hz

The actual value depends on the BSP and VxWorks configuration.

Once configured, the clock is enabled using:

sysClkEnable();

The system clock drives essential kernel services such as task delays, timeouts, scheduling-related timing, and other OS timing functions.

🖥️ VxWorks Image Types

#

Understanding the startup sequence also requires distinguishing between the major VxWorks image types.

BootRom Image

#

A BootRom image is a small bootloader-oriented VxWorks image.

Its primary responsibilities are to:

• Initialize essential hardware
• Establish enough operating-system infrastructure to load another image
• Access network or storage devices
• Load the main VxWorks image
• Transfer control to the target system image

Conceptually, it is similar to firmware or a BIOS-level boot environment rather than the final application operating system.

A BootRom may execute from:

• ROM
• NOR Flash
• RAM

depending on the BSP and target configuration.

It can also provide a lightweight multitasking environment containing services such as:

• usrRoot
• Network stack
• TFFS
• FTP
• Boot-device support

The corresponding build rules are typically defined through the BSP’s rules.bsp configuration.

VxWorks System Image

#

The VxWorks system image is the primary runtime image.

It contains:

• Kernel
• Drivers
• Filesystems
• Networking
• System services
• Application components

The image is commonly relocated or loaded into RAM, where its executable code, data, and BSS regions operate during normal runtime.

Its build behavior is controlled through the VxWorks image build configuration, commonly associated with rules.vxWorks.

The relationship can be summarized as:

BootRom
   │
   │ loads
   ▼
VxWorks System Image
   │
   ├── Kernel
   ├── Drivers
   ├── Filesystems
   ├── Network stack
   └── Applications

🔍 Complete ARM BSP Startup Sequence

#

Putting the individual stages together, the VxWorks 6.x ARM boot process can be represented as:

                    CPU RESET
                       │
                       ▼
                  _romInit()
                       │
             ┌─────────┴─────────┐
             │                   │
        Cold Boot            Warm Boot
             │                   │
             └─────────┬─────────┘
                       ▼
                 romInit()
                       │
             CPU / Memory / Stack
             Cache / Interrupts
                       │
                       ▼
                  romStart()
                       │
             Copy / Decompress
             ROM image into RAM
                       │
                       ▼
                   sysInit()
                       │
            Processor / Exception
             / Interrupt setup
                       │
                       ▼
                   usrInit()
                       │
             ┌─────────┼──────────┐
             │         │          │
           Cache      BSS      Exceptions
             │         │          │
             └─────────┼──────────┘
                       │
                       ▼
                  sysHwInit()
                       │
                Early hardware
                  initialization
                       │
                       ▼
                  kernelInit()
                       │
          Kernel / Stack / TCB setup
                       │
                       ▼
                  usrRoot task
                       │
             ┌─────────┼──────────┐
             │         │          │
          Memory      I/O      Drivers
             │         │          │
             └─────────┼──────────┘
                       │
                       ▼
                 sysHwInit2()
                       │
             Interrupt connections
             and late initialization
                       │
                       ▼
                 System Clock
                 sysClkEnable()
                       │
                       ▼
              Normal VxWorks Runtime

This sequence highlights the central design principle of VxWorks BSP initialization: each stage establishes the prerequisites required by the next stage.

🛠️ BSP Bring-Up Priorities

#

During initial BSP development, it is unnecessary to implement every peripheral immediately.

A practical bring-up strategy is to establish the smallest stable execution environment first.

The initial implementation of sysLib.c should focus on the core BSP functions:

sysModel()
sysBspRev()
sysHwInit()
sysHwInit2()
sysMemTop()

These functions establish the basic board identity, revision information, hardware initialization sequence, and usable memory boundaries.

Once the CPU can reliably boot, execute from RAM, initialize memory, and start the kernel, additional functionality can be introduced incrementally.

A sensible progression is:

CPU Reset
   │
   ▼
romInit()
   │
   ▼
Memory Initialization
   │
   ▼
romStart()
   │
   ▼
sysInit()
   │
   ▼
usrInit()
   │
   ▼
kernelInit()
   │
   ▼
usrRoot
   │
   ▼
Serial Console
   │
   ▼
System Clock
   │
   ▼
Interrupts
   │
   ▼
Storage / Network
   │
   ▼
Application Drivers

This approach makes failures easier to isolate because each newly introduced subsystem can be validated against an already functioning boot foundation.

📌 Key Takeaways

#

The VxWorks 6.x ARM BSP boot process is a layered transition from bare-metal processor startup to a fully operational real-time operating system.

The most important stages are:

The most important architectural distinction is between pre-kernel initialization and post-kernel initialization.

Before kernelInit(), the system must establish the processor, memory, exception, cache, and basic hardware environment without relying on normal multitasking services. After kernelInit(), usrRoot can progressively bring up the rest of the operating system and board peripherals.

For BSP developers, tracing execution from:

romInit()
    ↓
romStart()
    ↓
sysInit()
    ↓
usrInit()
    ↓
kernelInit()
    ↓
usrRoot
    ↓
sysHwInit2()

provides one of the clearest ways to understand why a VxWorks target fails during boot and which layer of the BSP needs investigation.

The practical rule is simple: get CPU and memory initialization working first, establish a stable kernel startup path second, and only then add interrupts, clocks, storage, networking, and application-specific drivers.