Radare2: 5.3.0 – Root Powder Goety

05/31/2021 10:20 pm

Radare2: 5.3.0 – Root Powder Goety

This release comes with a large list of bug fixes contained in 246 commits from the last 6 weeks thanks to 19 contributors. Kudos to everyone hanging out in the chats, testing, discussing, asking, helping and building up this community that makes r2 what it is. Hope all the users appreciate and enjoy this update as much as we did coding for it.

Greetings to: Alex Bender Anthoine Bourgeois condret David CARLIER Dennis Goodlett Giovanni Di Santi gogo2464 Jing Liu meme Michal Ambroz murphy pancake Rene Laemmert RHL120 Shadorain Siguza Simon Vareille StefanBruens Sylvain Pelissier

I could shout: aaaa is no longer breaking the debugged process! or Go scripting support!, but the list of changes and security bug fixes is quite large to summarize in just one line.

Some important bugs has been fixed in the build system, not just reflected in the README and the CI but also for both meson and acr, previous old installations of r2 no longer breaks the build. The rpath builds are now fixed for both acr and meson, this is required for r2env! also, and most important one, all the sdb databases are now precompiled in C and loaded at compile time instead of having to map disk files. This fixes the need to depend on side files installed in the system to make your static binary builds of r2 to work. This feature is now enabled by default and tested in the CI, but it can be also optionally disabled if you prefer the old behaviour which is more flexible. musl static builds are now officially supported and tested in the CI.

Friendly reminder that license documentation has been updated in doc/license.md and you can check at runtime all the licenses of the core and plugins used in your builds of r2 in case you need to care of such things.

Support for the S390 architecture and the z/OS architecture has been improved from RBin, RCharset and RAsm by adding support to extract ebcdic37 strings with rabin2 honoring cfg.charset and loading MVS OFF S/390 module objects, in addition, the latest S390 disassembler from GNU Binutils has been imported, which works side by side with the Capstone one.

Some important bugs has been fixed in the debugger. From infinite loops, fixed reg profile for arm64 debuggers, reseting the heap analysis on restart and other undefined behaviours that happened randomly on Linux and macOS. We greatly recommend you to update!

Multiline comments are better displayed on hexdumps and disasm, the order of flags and xrefs is now sorted to be more meaningful to the reader. The variable asm.sub.jmp is now working again. Other improvements with asm.meta=false for displaying data in the middle of code and better displaying of switch table comments. A new variable asm.hint.imm is now accessible from visual mode to pick immediates from instructions using hot keys. All those additions make visual and panels look even better!

The commandline have received some bold updates. The newshell parser has been removed from the codebase, which resulted in the following changes: improved commandline parser to fix all the tests working in newshell with the good-old-C-based parser, deprecate the backslash and single quote aliases for =! and promote the use of :. This is an important change for r2frida users!. The autocompletion tab is working again and has been extended to support more config var types. The whole refactoring end up with 30s less in CI builds and 512KB less sources.

New commands!

  • afxm : x/y map of function xrefs
  • wcf : write file contents + cache patches into given file
  • aev : the visual esil debugger (same as VbE)
  • aeis: initialize stack for given argc, argv, envp
  • x/w : long standing issue improves gdb-like examine commands in r2
  • ===stderr : allows to redirect r2’s stderr thru the new r2pipe.side api
  • px-- context hexdump command (like pd--)
  • : this always-undefined command is now replacing \\ and ' aliases

Command changes:

  • px now honors cfg.charset in the ascii column
  • pr : supports printing raw null bytes
  • Mark ' and \\ commands as deprecated. use :
  • Implement @@@e and @@@E to iterate over entries and exports

The RBin library ships with some important security bug fixes, covering some public CVEs for corrupted PE, Python and MACHO files. Additionally a cache has been added to greatly speedup the loading of DWARF files and adding support for two new file formats (OFF for zOS/S390 and WAD (the DOOM map files).

From the analysis perspective this release comes with some important changes: capstone5 is now the default disassembler and analysis library for most common architectures. The anal.calls variable is now honored in aa, which results in better code coverage when performing automatic analysis. Running aaaa no longer breaks the debugged process! There’s some little improvements in the type propagation analysis and the missing char** type is now included, which works in sync with the new aeis command to redefine the stack contents for a specific argc, argv, envp.

ESIL has deprecated the REPEAT keyword and extended the Thumb emulation by supporting the ldrd instruction, The arm64 assembler has been also improved a little bit warming engines for the r2wars. Non-intel users will also enjoy a more native experience along all the tools because.

A new IO plugin is available in default builds, the socket://, this plugin was implemented in r1.. but it never really reached r2 codebase until now! This plugin connects or listens to a tcp host:port and records a flag for every read operation that happens, writes are sent to the endpoint, this enables r2 to be used for protocol debugging, which can be easily scriptable with r2pipe for fuzzing or testing purposes. The old tcp plugin is now named tcp-slurp:// to avoid confussions.

Signature search, matching, storing and management has been improved, handling collisions of multiple metrics to better decide which match pick, bytes are now available as a metric for signature matching, diffing and comparison. This makes z/ run quiet faster and generate better results than before.

Better error messaging has been added in visual, panels as well in many commands like the infamous pf which use to spit confusing messages, now supports writing enums and bitfield values in mapped structs. Same goes for the pa command which now suggests pd in case the user mistypes it (as it seems to be from the feedback from users).

Summarized Highlights

  • removed newshell improved oldshell
  • switch to capstone5 and honor anal.calls for better code coverage and type propagation
  • initial support for analyzing s390/zOS module objects
  • \\ and ‘ aliases are now deprecated. Use =! or : from now on.
  • Improved ESIL with visual word level esil debugger for Thumb, arm64 and x86-64
  • Import socket:// from r1 for tcp network protocol debugging
  • Type information from the binary is now loaded by default
  • Improved stability of analysis and debugger on linux-arm64
  • Musl static bins with compiletime databases for better portability
  • Custom charset supported to find strings and hexdump ascii column
  • Disassembly listing improved for multiline comments and multiflag offsets
Changelog Highlights

Changes

anal

  • Improve aaaa log messages and avoid aaef to run in debugger mode
  • Fix many zero cases in some jump table analysis
  • Fix disasm alignment of data words in s390x disassembler plugins
  • Fix s390.gnu disassembly and add test for 6 byte instructions
    • Better debug messages instead of r_warn in jmptbl analysis
  • Better debug messages instead of r_warn in jmptbl analysis
  • Use gperf on anal/d and improve build and checks
  • Implement afxm command to show an call refs map
  • Apply fix in sixref plugin to be in sync with the latest xref
  • Type added: “char**” to SDB, ref #18633 (#18636)
  • Fix boundary check in aao to parse more refs
  • Fix ‘Cannot find return type for’ calling convention issues when saving a project (#18638)
  • Fix 13482 – Remove anal.jmp.after variable (#18629)
  • Improved type propagation analysis
  • Fix #18323 – honor anal.calls in aa

asm

  • Fix #18619 – Wrong assembly generated for: “add x0, x0, 1, lsl #12” (ARM64)
  • Fix rasm2 -w in termux (honor R_SYS_ARCH)
  • Fix rasm2 -w in termux (honor R_SYS_ARCH)
  • Handle je and jne as aliases for jz and jnz in wao
  • Rename sysz to s390 and add the s390.gnu plugin from binutils 2.36
  • Update sdb and use of SdbGperf in asm.d
  • Switch to Capstone5 as default

bin

  • Dont trust the unaligned rich PEs
  • Add initial toy IBM S390 Object File Format parser
  • Fix #18724 – Use RCharset in rabin2 -z
  • Always load bin types as pf. format strings
  • Add WAD file parsing (#18659)
  • Fix #18679 – UAF when parsing corrupted pyc files
  • Fix #18667 – division by zero in the macho parser
  • Speedup dwarf loading when no files are found in disk

build

  • Fix #12335 – ignore system-installed r2 includes
  • Add use_cgen meson option
  • Add support for acr/musl-gcc static builds
  • Improve the xxhash system library detection
  • Make meson compatible with older versions (RHEL8 meson 0.49) (#18684)
  • Fix –with-rpath and add CI tests (#18668)
  • Collapse all opcode_*.c files into opcode_all.c

charset

  • Implement IBM EBCDIC 0037 character encoding

cons

  • Fix grep cmd with neg (#18763)

core

  • Initial import of the ‘Radare2 Version Control System’ APIs (#18499)

debug

  • Add missing =SN and zf for the darwin-arm64 native debugger reg profile
  • Fix infinite loop in r2 -c ‘ood;ood’ –
  • Fix dmha output after ood (#18710)
  • Fix cast issue in ptrace call, waitpid fix
  • Show string version of the stop reason in di

diff

  • Add byte signature diff zd

disasm

  • Fix #18427 – Sort flags by [sections,formats][other][regs]
  • Fix multiline comments in ‘pd’ with asm.cmt.right=0 and 1
  • Avoid the use of sscanf, better parsing and error checking, handling negative switch cases
  • Fix #16677 – Honor asm.sub.jmp in pd
  • Fix #13200 – Honor anal hints in asm.meta=0
  • Swap xref and flag comments as suggested in #18427
  • Implement asm.hint.imm and integrate it in visual

emu

  • Add arm16 ldrd esil tests
  • Implement aeis to initialize argc, argv and envp for emulation

esil

  • Fix emulation for ARM’s ldrd
  • Add ‘aev’ as an alias for VbE and improve esil debugger
  • Fix #18736 – Eliminate REPEAT ESIL command, fix BSF/BSR x64 expressions

hash

  • Fix #18727 – Support more hashes in ph

io

  • Implement socket:// plugin, inspired by radare1
  • Rename tcp:// to tcp-slurp:// and improve help message
  • Add io.cache.nodup to not write the same bytes in the cache

lang

  • Add GO rlang plugin (#18646)

p\u2026

  • Properly report error when trying to use an invalid pd subcommand

print

  • Fix empty lines in hexdump with multiple comments in one line
  • Support multiline comments in px
  • Fix #18309 – Better error messages for the pf command
  • Fix #18308 – Fix pf parsing issues and support write on enums and bitfields
  • Add help messages for pde pdr pdp
  • Implement ‘px–‘ context hexdump command
  • Remove assertion in pFA
  • Fix #4903 – Handle ‘w’ in x/
  • Improve error message in pa command
  • Honor cfg.charset in px and support escaped encoding
  • Support null bytes in r_print_raw
  • Fix oob crash in ‘pri’ command

projects

  • Fix #18641 prevent overwriting projects with Ps

r2pipe

  • Add ===errmsg to support the new r2pipe side

search

  • Fix rafind2 issue with small or negative blocksize
  • Reset certificate search properly (#18664)

shell

  • Fix autocompletion for ‘e ‘ and handle cfg.charset=
  • Add initial rvc2 command
  • Fix #16674 – Kill ‘ and \\ commands, use ‘:’ for the only alias of =!
  • Implement @@@e and @@@e to iterate over entries and exports
  • Handle aliased files in wff and wtf commands
  • Remove tree-sitter and the r2-shell-parser

signatures

  • Consider collision
  • Fix bug in graph matching
  • Create function when z/ finds byte
  • Add R_SIGN_BYTES to metric search (#18703)
  • Support zignature collisions with the new zac command
  • Fix bugs in mergeItem zignatures
  • Refactor output and serialization of signatures
  • Make z/ search sigs seen in z*
  • Refactor signature matching
  • Add byte signature diff zd

tools

  • Add rafind2 -L to list IO plugins (same as r2 -L)

types

  • Fix #16687 – Handle multiple colon separated paths in dir.types

util

  • Add r_rbtree_cont_node_{first/last} (siol_eternal)
  • Add size parameter to r_magic_load_buffer

visual

  • Fix v;! in sync with V;!
  • Check if target is writeable in Vc+-
  • Add scr.optimize with experimental optimization ansi routine
  • Better v!!!!!!! behaviour
  • Add noflush guards to fix visual debugger mode

write

  • Implement wcf command to patch file with cache changes into a new file