ratgdo/homekit-ratgdo v1.9.0

on GitHub

v1.9.0 (2025-07-18)

A huge thank you to @frankea for all his contributions to this release that are expected to significantly improve the performance and stability of the HomeKit firmware.

Major Release - Comprehensive Stability and Performance Overhaul

This major release represents a comprehensive rewrite of critical stability components with extensive testing infrastructure. All known crash conditions have been eliminated and performance has been dramatically improved.

Critical Stability Fixes

• Fixed 6 Critical Race Conditions - Eliminated all identified critical failure modes that could cause system crashes, permanent hangs, or data corruption
• Millis() Rollover Safety - Fixed timing bugs that caused permanent system hangs every ~49.7 days of uptime
• ESP8266 Alignment Crashes - Fixed Exception 9 crashes by ensuring 4-byte alignment for multi-byte data structures
• Buffer Overflow Prevention - Fixed Exception 0 crashes from stack memory corruption with comprehensive bounds checking
• Interrupt Safety - Fixed race conditions in obstruction sensor ISR that caused false readings
• Stack Overflow Protection - Prevented crashes in dense WiFi environments through safe array sizing
• Configuration Corruption - Fixed race conditions in config writes that could corrupt settings during WiFi events
• Rolling Code Protection - Eliminated race conditions that could desynchronize door opener communication

New Features

• Comprehensive Testing Framework - Added Unity-based test suite with 11 test categories covering all critical functionality
• Smart Obstruction Detection - Automatic fallback from pin-based to Pair3Resp packet-based detection when hardware sensor unavailable
• Performance Monitoring - Real-time web performance metrics exposed via JSON API (requests, cache hits, dropped connections, max response time)
• Enhanced Security+ 1.0 Support - Improved door state validation and reduced "Door State: unknown" occurrences
• Memory Usage Tracking - Comprehensive monitoring of both regular and IRAM heap usage
• Static Analysis Integration - cppcheck integration for continuous code quality monitoring
• GitHub Actions CI/CD - Added .github/workflows/test.yml and codeql.yml for automated testing and security scanning
• Test Infrastructure - Added run_tests.sh script and comprehensive test/README.md documentation
• Performance Profiling - Added test_performance.py for measuring real-world response times

Performance Improvements

• 68% Faster Web Interface - JSON caching reduces response times from 459ms to 146ms
• 277% More Free IRAM - Optimized memory usage from 1.9KB to 7.3KB free
• Connection Management - Throttling, timeout protection, and resource leak prevention
• WiFi Stability - Stack overflow prevention in dense network environments (20 network limit)
• Long-term Reliability - Rollover-safe timing for continuous operation beyond 49 days

Testing Infrastructure

• Unity Test Framework - 11 comprehensive test suites with 100% pass rate
• Core Functionality Tests - Rollover safety, validation logic, and ESP8266-specific behavior
• Integration Tests - HomeKit functionality and protocol communication
• Performance Tests - Memory usage, timing analysis, and resource monitoring
• Hardware Simulation - Door operation testing with mocked hardware interfaces
• Web Interface Tests - REST API endpoints and response validation with Python unittest
• Static Analysis - Automated code quality checks with cppcheck
• Build Monitoring - Size tracking and memory usage validation
• CI/CD Integration - Automated testing on all commits and pull requests
• CodeQL Analysis - Security and vulnerability scanning for C++, Python, and JavaScript
• Test Documentation - Comprehensive test/README.md with coverage goals and debugging guides
• Mock Hardware Layer - Simulates ESP8266 memory functions and Arduino framework for native testing

Memory and Performance Optimizations

• RAM Conservation - Removed duplicate JSON buffer saving 1.3KB RAM for memory-constrained ESP8266
• IRAM Optimization - Strategic buffer placement providing 5.4KB additional memory headroom
• Connection Throttling - Max 4 concurrent connections with 5-second timeout protection
• WiFi Stack Safety - Limited network scanning to prevent overflow in dense environments
• Safe String Operations - Bounds-checked string concatenation preventing buffer overflows
• Request Caching - JSON response caching dramatically improves repeat request performance
• Memory Monitoring - Real-time tracking of heap fragmentation and usage patterns

Technical Implementation Details

• Struct Alignment - Added __attribute__((aligned(4))) to PacketAction and ForceRecover structs preventing Exception 9 crashes
• Rollover-Safe Arithmetic - Replaced all direct millis() comparisons with rollover-safe subtraction patterns ((int32_t)(millis() - last_time) > timeout)
• Interrupt Safety - Protected pulse counter access with proper synchronization between ISR and main loop using noInterrupts()/interrupts()
• Buffer Management - Replaced Variable Length Arrays with fixed-size arrays for stack safety
• String Safety - Replaced unsafe strcat with bounds-checked safe_strcat wrapper functions
• Type Safety - Added proper format specifiers (ADD_LONG, ADD_TIME macros) eliminating compiler warnings
• Memory Layout - Strategic buffer allocation between IRAM and regular heap for optimal performance (LOG_BUFFER_SIZE in IRAM heap)
• Connection Management - Web server rate limiting, timeout handling, and resource leak prevention
• Protocol Enhancement - Pair3Resp packet parity detection (3=clear, 4=obstructed) for obstruction fallback
• State Validation - Enhanced Security+ 1.0 logic accepting valid states immediately while confirming suspicious values
• Config Protection - Atomic file writes with mutex protection preventing corruption during concurrent access
• Config Write Optimization - Added configChanged flag to prevent unnecessary flash writes on transient operations
• Network Safety - WiFi scanning limits (MAX_NETWORKS=20) and stack overflow protection in dense environments
• Request Throttling - ActiveRequest tracking with MAX_CONCURRENT_REQUESTS=4 and REQUEST_TIMEOUT_MS=5000
• Diagnostic Logging - Added comprehensive RINFO/RERROR logging for debugging race conditions and system state
• Memory Allocation Safety - Added malloc failure protection with automatic ESP.restart() in log.cpp and utilities.cpp
• Atomic Configuration Writes - Implemented temp file + rename pattern to prevent config corruption during power loss
• Config File Validation - Added malformed line detection and graceful handling in config parser
• WiFi Connection Optimization - Reduced connection delay (500ms→100ms) and added 10-second timeout protection
• WiFi Stack Protection - Dynamic memory allocation for network lists and MAX_NETWORKS=20 limit preventing overflow
• Code Quality - Removed trailing commas in HomeKit characteristic declarations for compiler compliance
• LOG_BUFFER_SIZE Optimization - Reduced from 8192 to 2048 bytes saving 6KB IRAM for critical services

Issues Resolved

• #124 - Obstruction sensor unreliable/always shows obstructed (fixed by automatic fallback to Pair3Resp packet-based detection)
• #132 - Security+ 1.0 door state synchronization issues and frequent "Door State: unknown" (fixed by improved state validation logic)
• #218/#215 - Memory-related crashes and HomeKit malloc failures (improved by IRAM optimization and connection management)
• #252 - SEC+1.0 bootloop crashes due to IRAM heap exhaustion during HomeKit MDNS initialization (fixed by LOG_BUFFER_SIZE optimization)
• #261 - Timing issues and bugs after millis() rollover (49+ day uptime)
• #266 - Slow web interface performance and timeouts
• #267 - Connectivity crashes, web interface timeouts, WiFi instability, Exception (0) crashes with ASCII in addresses
• #271 - ESP8266 alignment crashes (Exception 9/LoadStoreAlignmentCause) due to unaligned struct access