github rizinorg/rizin v0.5.0
Release v0.5.0

latest releases: v0.7.3, v0.7.2, v0.7.1...
19 months ago

We are happy to present the result of our half-year long work - v0.5.0 release. Within this release we almost finished (only a few commands remain) the conversion to the new tree-sitter-based parser. Moreover, we also progress with the RzIL migration - three more architectures were converted since the last release: PowerPC, SuperH, and 8051. We also started the conversion of the x86 - most basic instructions are covered. One more significant change since - is the use of a new frame-pointer-based variables addressing instead of the old SP/BP-based ones.

As usual, keep reading for more insights and let us know of any bugs you find or improvements you would like to see in future versions.

New

  • With this release we completely removed the old bash-based rz-pm package manager to focus on the new cross-platform package manager, rz-pm, written in Go and easy to install on all major OS and platforms.
  • New color themes: Nord (eco nord) and Mars (eco mars).

Screenshot 2023-02-16 at 16 36 02

Screenshot 2023-02-16 at 16 36 36

  • Stack pointer tracking: The stack pointer was previously already tracked during analysis for variables and their accesses, but not stored in any sensible form for further use. Starting with this release, it is stored in a way that allows for efficient querying of the stack frame size value at arbitrary analyzed addresses. The asm.stackptr variable can be enabled to visualize both the absolute sp value and the delta of each instruction in disassembly:

  • Stack-based variables: Variables on the stack are not identified by bp/sp+ anymore, which often caused misleading results, but by their address from the bottom of the stack frame, independent of how they are accessed.
    Notice how in this example from rizin 0.4.1, the analysis creates both bp- and sp-based variables from DWARF info and disassembly respectively, but is unable to detect that they actually describe the same stack locations:

    With the new kind of addressing (displayed as stack - <offset>), such cases are resolved:

    This also enables decompiler plugins like rz-ghidra to integrate sp-accessed variables defined in rizin in their decompilation output.

Improvements / Fixes

  • All of our tests now pass on both Little Endian and Big Endian platforms. For a long time we had tests failing when executed on Big Endian systems, due to subtle assumptions the running system was Little Endian. Not anymore! We do have S390x machines running our test suite on every commit to ensure that we won't regress anymore.
  • String search is now multithreaded! Now multi-core processors can use parallelism to find string across the binary which results in faster loading time.
  • FLIRT generation and detection had important fixes covering cases that were wrongly handled before. You can expect much better detection now and better coverage with our sigdb (this also improves the compatibility with IDA)
  • cabextract tool presence is no longer necessary for unpacking PDB files, as rizin is able unpacking them itself using the "libmspack" library.

Extras

Packaging

  • libuv dependency was removed
  • liblzma dependency was added
  • libmspack dependency was added
  • blake3 dependency was added

Build

  • Remove old bash-based rz-pm (#3360)
  • Update xz (liblzma) to 5.4.1
  • Update liblz4 to 1.9.4
  • Update zlib to 1.2.13
  • Update tree-sitter to 0.20.7
  • Use compiler intrinsics for rz_swap_* functions (#3300)
  • Fix plugin removal and clean code (#3277)
  • Removed rzbochs
  • Update blake3 to use cpu extensions (#3108)
  • Remove rz-agent (#3087)
  • Add ssdeep hash (#3084)
  • build: fix BINDIR_DEPTH computation on Windows (#3075)
  • Replace meson cmake module usage by plain configure_file() (#2948)
  • Meson refactor: collect modules in dict and centralize config (#2945)
  • Build: add option to choose to regenerate cmd_descs.[ch] (#2934)
  • Rename rz_version.h.in to rz_build_version.h.in (#2914)
  • Fix Windows cross build (#2897)
  • Fix windows and cpp compatibility (#2887)
  • Drop libuv dependency
  • Update on every build the git hash shown by rizin -v (#2863)
  • librz: refactor a bit how plugins are built
  • util: make zlib dependency optional
  • Set default warning level 1 for Meson
  • Update libzip to v1.9.2
  • build: keep using meson 0.55.0 functions
  • build: use gnu99 globally instead of per-project
  • Add liblzma subproject
  • Add libmspack and add idpx to extracts compressed pdb (#2728)
  • build: check for 'backtrace' presence in meson
  • Fix compilation on Termux and Haiku (#2671)
  • Use clang-format 15 instead of 14

RzUtil

  • Fix plugin removal and clean code (#3277)
  • Add *pty API (#3221)
  • Fix: grep(~) will change the original result when ignores case (#3259)
  • util: fix logging of long strings by using RzStrBuf
  • Remove RZ_HOMD_OLD_PLUGINS after transition period ended (#3246)
  • Refactoring and rename RASN1 to RzASN1
  • Rename RPKCS7/RCMS/Spc to RzPKCS7/RzCMS/RzSpc
  • Refactoring names and usages of pkcs7
  • Rename RX509* to RzX509*
  • Refactoring names and usages of x509
  • Refactoring RzList to remove unused methods and bad naming (#3074)
  • Added Apple ASN1 OIDs (#3048)
  • Fix oob read on rz_pkcs7_parse_spcdata
  • Fix oob read on rz_x509_parse_tbscertificate
  • Add rz_bv_range_set and rz_bv_is_all_one to bitvector lib (#2891)
  • Move rz_str version funcs into version.c (#2890)
  • Remove cb_printf() from RzPrint
  • rz_print_hexdump() refactoring (#2841)
  • Tokenize and colorize asm strings (#2417)
  • Fixes for the regex engine (#2839)
  • Windows: Subprocess fixes
  • Windows: Refactor environment variables
  • Windows: Implement rz_sys_pipe()
  • Windows: Fix for file sharing permission
  • util/sdb: remove JSON code
  • Move SDB into RzUtil
  • util: make zlib dependency optional
  • util: provide rz_time_gettimeofday API (#2811)
  • Remove rz_th_kill, rz_th_kill_free and rz_th_pool_kill (#2790)
  • Windows: Fix for subprocess API handle inheriting (#2773)
  • Add RzBaseFindOpt for custom thread status callback (#2768)
  • Multithread string search (#2665)
  • Remove RQueue (#2720)
  • Cleanup of utf8.c and utf16.c files. (#2715)
  • Rework register profile parsing
  • Refactor thread code and add RzThreadQueue (#2683)
  • Print: Memoize number of rows for current print
  • Better false-positives detection in rz_scan_strings (#2691)

RzType

  • Make afs respect the calling convention (Fix #3291) (#3292)
  • Remove temporary type string conversion in DWARF integration (#3266)
  • Fix type bitsizes for struct/union member loops
  • Fix rz_type_db_get_bitsize() without specifier and add tests
  • Rewrite and extend type paths from offset
  • Unify and fix typedef loop handling
  • Fix leaks in rz_type and tests (#3223)
  • Ensure double and float uses the proper method to decode it. (#3105)

RzSignature

  • Do not overly optimize when applying a pat file
  • Fix signature creation due bad offset
  • Rename only non-symbols, but add the flag
  • Fix FLIRT CRC calculation, length and matching
  • Fix FLIRT detection when function size is less than 32 bytes (#3118)
  • Remove zignatures and move aaF[l] -> F[al] , zf[sdc] -> F[sdc] (#2682)

RzIO

  • Return error when reading/writing to unmapped memory (#3323)
  • Refactor S-Record files reading to not use sscanf() (#3321)
  • Rewrite of Bochs debugger and io plugin
  • Rewrite reading in io_mach (#3101)

RzHash

  • Fix alignment issues in SHA2
  • Add ssdeep hash (#3084)
  • Support blake3 hash algo (#3030)
  • hash: use RzHash in most hash APIs
  • RzHash: rename everything in librz/hash to RzHash prefix

RzDemangler

  • Add Free Pascal demangling support (#3011)

RzDebug

  • Fix DMP reg profile depending on host bitness
  • Make XNU reg profile depend on CPU type instead of RzDebug.bits (#3270)
  • Fix endianness for Windows PageDump files and WinKd (#3204)
  • Adds sync_registers in debug plugin and cleanup brainfuck debugger
  • Fix up rz_debug_trace_ins_after() (#3216)
  • rz_debug_trace_ins_after: Downgrade dbg->cur_op guard (#3206)
  • Print err msgs if rz_debug_trace_ins_before() fails (#3203)
  • Fix arm64 register access in XNU debugger
  • Use cpu_type_t for checking XNU debuggee's arch
  • Refactor global state of xnu debugger into RzXnuDebug struct
  • Refactor and replace .dm* calls with pure C (#2847)
  • Fix rz_xnu_get_cur_thread() messing up the target's mappings
  • Fix detaching in XNU Debugger
  • Fix attaching in XNU Debugger
  • Improve logging for EXC_SOFTWARE on XNU, showing Unix signals
  • De-include xnu_threads.c and xnu_excthreads.c
  • Remove XNU_USE_PTRACE
  • Fix pdb.autoload crash (#2729)

RzCons

  • Mute the mouse in the HUD mode (#3355)
  • Fix big memory leak in graph rendering (#3342)
  • Add search.case_sensitive option for grep case-sensitive/insensitive (#3241)
  • Fix: grep(~) will change the original result when ignores case (#3259)
  • Add Nord color theme (#3250)
  • Add Mars color theme (#3252)
  • Extract histograms and bars into RzCons (#3189)
  • visual: extract classes from vmenus.c to classes.c
  • Windows: Remove rz_line_readchar_win()
  • Windows: Fix rz_cons_isatty() for COM and NUL
  • Windows: Fix dietline under mintty
  • Windows: Fix VT mode support detection
  • Windows: Use ReadFile if stdin is not a console
  • Fix reset followed by color change in rz_cons_html_filter (#2912)
  • Fix the chaos while clicking in panel mode (#2832)
  • Windows: Completely save & restore console state
  • Cons: Memoize current context row/column number

RzBin

  • Fix #2859 - Use minstring in all bin plugins (#3328)
  • Fix #3128 - Add overflow checks on dyldcache bin values. (#3303)
  • Patch Mach-O chained ptrs into sparse overlay RzBuffer
  • Fix searching for strings in rz-bin when the file format is invalid (#3245)
  • bin/formats/elf: consider stripped bins without sections (#3248)
  • Fix endianness for win pagedump and winkd (#3204)
  • Fix NE reloc endianness bug (#3224)
  • ELF: added vn_next break condition (#3214)
  • Support for Xbox 360 PE32 architecture (PPC BE) (#3168)
  • Fix endian in macho reloc parsing
  • Fix endianness when parsing PE trycatch
  • Revert back wrong fix due now fixed endianness in rz_str_utf16_to_utf8
  • Fix endianness in PE exports
  • Fix mdmp endianness (#3126)
  • Fix luac endianness (#3123)
  • Fix coff on big endian hosts (#3122)
  • Refactor pe_debug.c to fix endianness (#3121)
  • Fix dyldcache slide version reading on big endian
  • Fix ninds plugin endianness and improved it
  • Make dyld_chained_ptr_* parsing endian-independent (#3113)
  • Resolve some BE issues on various files
  • Refactor bin_mbn.c to fix endianness and remove global
  • Rewritten bin_nin3ds.c and add test (#3110)
  • Fix endianness in pyc (#3107)
  • Fix parsing QNX format on BE platforms (#3098)
  • Mach-O: Add support for DYLD_CHAINED_PTR_64 (#3094)
  • Fix #3031 - Fix div by zero in dyldcache_rebase.c
  • Fix #2993 - Check rz_buf_read_le32_offset return status parsing LE bins (#3005)
  • fix #2970 - oob read in coresymbolication.c and in bin_dyldcache.c
  • fix #2966 #2967 #2974 - oob read in asn1.c
  • fix #2972 - oob read in ne.c
  • fix #2971 - null deref dwarf_process.c
  • fix #2970 - oob read in pe/pe_info.c and bin/golang.c
  • fix #2969 - oob write (1 byte) in dex.c
  • fix #2968 - null deref in mdmp.c
  • fix #2965 - null deref and div by zero in mach0_rebase.c
  • fix #2964 - double-free in bin_qnx.c
  • fix #2963 - oob write (1 byte) in pyc/marshal.c
  • fix #2962 - oob read in bin.c
  • fix #2961 - oob read in coresymbolication.c
  • fix #2959 - oob write in bin_dyldcache.c
  • fix #2958 #2960 #2973 - oob read in dwarf.c
  • fix #2957 - oob read in pe_section.c
  • fix #2956 - oob write in mach0.c
  • fix #2955 - oob read in dex.c
  • fix #2954 - oob read in dex.c
  • fix #2952 - null deref in dyldcache.c
  • refactor bin_symbols code to detect bad values.
  • fix integer overflow in mach0
  • fix oob read on luac
  • Handle endianess during NE format parsing (#2941)
  • Handle endianess during MZ format parsing (#2937)
  • Resolve golang string structures in data section (#2878)
  • Fix endian issues and libs in xbe and extend tests (#2843)
  • Fix reading VS_FIXEDFILEINFO on big endian
  • remove Ldr plugins
  • Remove globals from mach0 code
  • Remove globals from pyc code
  • Remove usage of rz_bin_is_big_endian (#2761)
  • bin/p/ELF: convert to INFO log some WARN
  • bin/p/ELF: parse gnu_debugdata section
  • Add RzBinStrDb and api to interact with it via rz_bin_object_string_* (#2754)
  • Add libmspack and add idpx to extracts compressed pdb (#2728)
  • Split RzBin PE parser into smaller pieces
  • RzBin/PE: fix parsing of malformed PE with 0 FileAlignment (#2722)
  • Remove RZ_BIN_STRING_ENC_* in favor of RZ_STRING_ENC_* (#2716)
  • Add dotnet PE support (#2528)
  • Better false-positives detection in rz_scan_strings (#2691)
  • Add EM_KVX elfid (#2690)
  • Fix default ELF arch for x86 binaries
  • Fix endian issue and unaligned access in PDB (#2667)

RzAsm/RzAnalysis

  • Add instructions introduced in Hexagon v70, v70t and v73. (#3258)
  • x86 RzIL uplifting (#2747)
  • Update librz/asm/arch/cris/gnu/ to the latest binutils (#3127)
  • Fix #2851 - wrong AVR analysis jump value for rjmp/rcall (#3012)
  • Fix tms320 disassembler for big endian hosts
  • Fix asm_riscv on big endian hosts (#2918)
  • Fix arm64 assembler tests on big endian hosts (#2913)
  • [Hexagon] Resolve duplex instructions dynamically (#2882)
  • [RzIL] IL lifting of SuperH ISA (#2518)
  • Tokenize and colorize asm strings (#2417)
  • Fix pic18c disassembler on big endian
  • Replace bp/sp-vars by generic stack-based ones (#3167)
  • Rewrite stack pointer tracking and storage (#3207)
  • Fix endianness in analysis mips.gnu plugin
  • x86 RzIL uplifting (#2747)
  • Threaded analysis matching (#3090)
  • PPC uplifting to RzIL (#2823)
  • Rename rz_arch to rz_platform (#2708)
  • Add analysis.jmp.tblmaxoffset configuration option

RzShell

  • core/cmd: return CMD_STATUS_OK even when env var is not found
  • core/cmd: handle properly errors while executing commands substitution
  • Convert pr commands to the rzshell
  • Convert p= commands to rzshell
  • core: implement simple 'help' command (#3319)
  • Convert pC commands to rzshell (#3213)
  • core/cmd: fix 'sleep' command to consider whole arg
  • Fix wB test due broken pv1 (#3225)
  • Convert p- commands to rzshell (#3198)
  • Convert pi commands to rzshell (#3187)
  • Convert pk, pK, and pkill commands to the rzshell (#3178)
  • Convert po commands to the rzshell
  • Convert pp commands to the rzshell (#3112)
  • Convert pI commands to the rzshell
  • Remove unimplemented pA command
  • Convert pu commands to the rzshell
  • Convert pv commands to rzshell (#3069)
  • Convert dk commands to the rzshell (#3078)
  • Convert de commands to the rzshell (#3083)
  • Convert pF commands to the rzshell (#3071)
  • Remove agd and add rz_analysis_similarity/match + refactoring
  • Convert dW command to the rzshell
  • Convert dw command to the rzshell
  • Convert dg command to the rzshell
  • Convert dd commands to rzshell (#3049)
  • Convert dx commands to the rzshell (#3043)
  • Convert dp commands to the rzshell (#3047)
  • Convert di commands to the rzshell (#3041)
  • Convert some of the a commands to the rzshell (#3025)
  • core/cmd: rewrite rz_cmd_macro API
  • core/cmd: remove some unnecessary fields for macro handling
  • core/cmd: remove brk concept in macros because unused
  • core/cmd: remove RZ_API and rz_ pfx from non-exported functions
  • Expose pda and fix memory leaks (#2943)
  • Convert ag to rzshell (#2916)
  • core/cmd: fix CHOICES argument selection between choices_cb/choices (#2932)
  • core/cmd: print graph when dmhg is executed (#2931)
  • Fix pg with fewer than 5 arguments (#2921)
  • core/cmd: fix afx command JSON output
  • Convert px[ dDhHwWqQol] to rzshell
  • Port command pb/B to new shell (#2828)
  • Refactor and replace .dm* calls with pure C (#2847)
  • Remove lcmds (#2848)
  • Update embedded fd command (#2838)
  • Remove command aC (#2793)
  • Port command pa to new shell (#2824)
  • Port command p6 to new shell (#2819)
  • Add psc command to generate a C/C++ string (#2791)
  • Add hints for fe/f commands (#2785)
  • Convert command ai to new shell (#2778)
  • Convert command aL(j) to new shell
  • Fix the crash of command fg (#2758)

Full Changelog: v0.4.1...v0.5.0

Don't miss a new rizin release

NewReleases is sending notifications on new releases.