В одном из чатов подсказали &str в качестве типа downcast использовать. Что как оказалось логично, тк ловушка для panic содержит &str либо String:
https://doc.rust-lang.org/std/macro.panic.html :
The behavior of the default std hook, i.e. the code that runs directly after the panic is invoked, is to print the message payload to stderr along with the file/line/column information of the panic!() call. You can override the panic hook using std::panic::set_hook(). Inside the hook a panic can be accessed as a &dyn Any + Send, which contains either a &str or String for regular panic!() invocations. To panic with a value of another other type, panic_any can be used.