Another issue is that SmartMatrix, while better than all the other libraries on arduino, doesn't support all kind of weird panels out there, specifically the AB and AC panels that you often end up getting when you get higher resolutions like 128x64.

Using Neopixels would work better of course, but they caan't reasonably be had in less than P5 (0.5cm/LED) while RGBPanels go down to P2. Also, neopixels are about 10X more expensive per pixel, if not more.
So, the solution is to use a rPi to drive RGBPanels of size 128x128 and larger (a single Pi with 3 parallel channels can reasonably run up to 256x256. After that it gets harder and you need multiple microcontrollers).

This is where the excellent https://github.com/hzeller/rpi-rgb-led-matrix driver comes in. It's the most feature complete RGBPanel driver for microcontrollers

Ok, but you have all this arduino code, maybe written for a FastLED::NeoMatrix or Adafruit::NeoMatrix array using the Adafruit::GFX API. Or maybe, you used the FastLED API with an XY mapping function, or maybe even you're using the LEDMatrix API for FastLED. None of those work on rPi, and you don't want to change/rewrite your code.

Well, there is good news: you can use https://github.com/ChrisMicro/ArduinoOnPc to run arduino code on PCs, and therefore also rPi. It is however designed to display in an X11 windows, which is not what you'd want. So, instead, I forked it for you and wrote a rPi glue driver for my FrameBuffer::GFX base class: https://github.com/marcmerlin/ArduinoOnPc-FastLED-GFX-LEDMatrix

You therefore end up getting access to those 3 arduino graphics APIs, and you can render on rPi using a much faster and the more feature complete https://github.com/hzeller/rpi-rgb-led-matrix driver

And this is what the end result, looks like with a cool resolution of 128x192 with 3 panels of 128x64 run in parallel with the active-3 board from https://github.com/hzeller/rpi-rgb-led-matrix/tree/master/adapter/active-3

By using these driver options, I get about 400Hz refresh rate on rPi3, lowering the amount of pwm bits:
~/rpi-rgb-led-matrix/examples-api-use/demo --led-gpio-mapping=regular --led-rows=64 --led-cols=128 --led-row-addr-type=0 --led-chain=1 --led-show-refresh --led-slowdown-gpio=1 --led-parallel=3 --led-pwm-dither-bits=1 --led-pwm-bits=7 --led-panel-type=FM6126A -D0

Those variables are assigned when you create "rgb_matrix::Canvas *canvas", which is fed into matrix->setCanvas(). See this example code:

With rPi, especially with the active-3 board you can drive resolutions of at least 256x256. Here is an example of 128x192, 4 times bigger than the highest resolution I was ever able to drive on ESP32 (64x96). Here is a video natively playing on rPi:

Well, actually 2 months later, I was able to get to the probably max achieveable resolution, 384x256, see RGB Panels, from 192x80, to 384x192, to 384x256 and maybe not much beyond

384x192 resolution with 9 panels

384x256 resolution with 12 panels