What about drawing in bitmap with low latency and building the vector drawing from buffered user input in the background? Then replace bitmap with vector when it's available. It could be low latency and vector drawing at the same time.