Once you have video out, Memory eating options become quite available. Framebuffers are quite large.

I've got a 1.4inch screen waveshare rp2350 sitting beside me at the moment. I'm currently fighting getting paletted display update via DMA with limited success. As I work through it it does become apparent to me that you could generate a tiled+sprites mode a scanline at a time on demand with PIO and DMA. I'm not convinced it could be run with zero CPU involvement, but it could certainly be done with very little CPU.

For 640x480 output not overclocked, I estimate tile + sprite CPU utilization to be about 50% of one core. Of course you have two cores. That number goes up and down depending on resolution, particularly when you're pixel doubling.

It's absolutely doable. There's the beginning of a tile demo (a scrolling brick wall) in the pico-dvi-rs repo.

I was thinking of doing almost everything in PIO + DMA

Here's the gist of it, I haven't implemented it so there will be devils in the details but I think it's doable.

One DMA (1) outputting the current line(unless pixel format change is done at this stage as well)

A bunch of chained DMA to assemble the next line almost like a display list. Sprites/tiles are entries in the display list.

The list entries contain the DMA/PIO setups to manage source, destination, pixel format etc. A DMA(2) transfers a list entry to another DMA(3) to write the required fields a list entry is essentially a series of [destination]<-constant or [destination]<-FIFO. DMA(3) just gets the job of being told by DMA(2) where to put everything. That is sufficient (I think) to set up the other DMA channels arbitrarily from the list data.

Pixel format conversion is done by a PIO writing some number of bits out as a value that gets DMA'd to the low byte of a DMA source register and then triggering a DMA transfer with a length of 1 to read the palette entry and copy the color information into another PIO FIFO. I'm not sure on the best way to do sprite transparency, There's definitely ways to suppress writing on source==0. Would need investigating, worst case scenario would be conditionally triggering one of DMA [destination++]<-color or DMA [destination++]<-[destination]

Once the display list is set up for a mode it wouldn't be touched. All of the dynamic parameters would come from the locations read by list entries. Tile maps, tile images, sprite locations, sprite data etc would not be in the master list, just the locations of where to find those.

Then all that remains is to make a toast to the memory of Jay Miner.

I want to use the 2nd core to a evaluate a command buffer per line with linewise double buffering instead of keeping a whole frame buffer around. With commands to draw lines, boxes, curves, bitmaps, text etc.

We're very much thinking along similar lines. I also have an idea for a 4bpp image encoding that could be fast enough for streaming from SD card, and high quality, given preprocessing.