Whiteboards are designed to work smoothly in most popular use cases, however you might encounter a problems with performance. Please follow those guidelines to avoid most common issues when designing your whiteboard content.
Avoid animated gifs, or heavy weight iframes, videos and images - if really needed, put them in a dedicated section of the board.
Design the content so that it's mostly useful on 100% zoom
Don't concentrate too much content in the same area of a whiteboard, spreading it around infinite canvas will certainly improve the performance. A rule of a thumb would be to keep at least one screen(at 100% zoom) distance between logical chunks of content.
Turn off the mini-map
Split the board into multiple ones (as a backup solution, just in case whiteboards fail to work smoothly on a single one)