Mouse move causes jittering

Hi,

When i move the mouse in my (Windows Desktop) app there is a noticeable jittery appearance to the moving artwork on screen that stops when the mouse is still.

There is no noticeable drop in frame-rate, instead it just seems that moving the mouse causes the rendering to jitter.
I have a lot of images and animations(spritesheets) being drawn on screen but my apps frame-rate is 59-60 frames per second regardless of whether i move the mouse or not.

I am NOT overriding the mouse move virtual function in appbase so not sure how mouse move could be affecting my app. virtual void mouseMove( MouseEvent event ) {}

And i am NOT subscribing to the signal getSignalMouseMove in the window class.

Is there some way that cinders mouse move code could affect drawing in my app?

Can we take a look at the code?

Bala.

Heres my app code…

using namespace ci;
using namespace ci::app;
using namespace std;

class MyApp: public App {
public:
	static void prepareSettings(Settings* pSettings);

	void setup();
    
#if ( defined CINDER_MSW || defined CINDER_MAC )
	void keyDown( ci::app::KeyEvent event );
	void keyUp( ci::app::KeyEvent event );
	void mouseDown( MouseEvent event );
	void mouseUp( MouseEvent event );
	//void mouseMove( MouseEvent event );
	void mouseDrag( MouseEvent event );
#endif        

	void update();
	void draw();
	void resize();
    
#if ( defined CINDER_MSW || defined CINDER_MAC )
    CGameAppPS m_GameAppPS;
#endif
};

void MyApp::prepareSettings( Settings *settings )
{
#if ( defined CINDER_MSW || defined CINDER_MAC )
	settings->setBorderless(false);
	settings->setFullScreen(false);
    settings->setResizable(true);    
	settings->setWindowSize(1280, 720);

    
#ifdef CINDER_MAC
    //  fixes the problem with dropping framerates
    settings->disableFrameRate();
    ci::gl::enableVerticalSync( true );
#endif

#ifdef CINDER_MSW

	//settings->setFullScreen(false);
	//settings->setWindowSize(800, 600);

	//settings->disableFrameRate();
	ci::gl::enableVerticalSync(true);
#endif

#endif
    
}

void MyApp::setup()
{
    
#if ( defined CINDER_MSW || defined CINDER_MAC )
    m_GameAppPS.Setup();
#endif
        
}

#if ( defined CINDER_MSW || defined CINDER_MAC )
void MyApp::keyDown( ci::app::KeyEvent event )
{
	if( event.getCode() == ci::app::KeyEvent::KEY_ESCAPE ) {
		quit();
	}
    
	if( event.getCode() == ci::app::KeyEvent::KEY_f ) {
		setFullScreen(!isFullScreen());
		m_GameAppPS.EmitToggleScreenSignal();
	}
    
    m_GameAppPS.KeyDown( event );
}

void MyApp::keyUp( ci::app::KeyEvent event )
{
	m_GameAppPS.KeyUp( event );
}

void MyApp::mouseDown( MouseEvent event )
{
	m_GameAppPS.MouseDown( event );
}

void MyApp::mouseUp( MouseEvent event )
{
	m_GameAppPS.MouseUp( event );
}

//void MyApp::mouseMove( MouseEvent event )
//{
	//m_GameAppPS.MouseMove( event );
//}

void MyApp::mouseDrag( MouseEvent event )
{
	m_GameAppPS.MouseDrag( event );
}
#endif

void MyApp::resize()
{
    
#if ( defined CINDER_MSW || defined CINDER_MAC )
    m_GameAppPS.Resize();
#endif
    
}

void MyApp::draw()
{
	// clear out the window with black
	gl::clear( Color( 0, 0, 0 ) );
    
#if ( defined CINDER_MSW || defined CINDER_MAC )
    m_GameAppPS.Draw();
#endif
    
}

void MyApp::update()
{
    
#if ( defined CINDER_MSW || defined CINDER_MAC )
    m_GameAppPS.Update();
#endif
    
    
}

CINDER_APP(MyApp, RendererGl, MyApp::prepareSettings)

Hi,

only thing I can think of is that the GPU driver’s vertical syncing might be implemented as a busy-wait (this is the case for NVIDIA GPU’s in general). This means that your main thread will block completely at every buffer swap, outside of your control. Therefor, if you have vertical sync enabled, your main thread can only process events a limited amount of time per frame and needs to catch up after each frame. Because mouse move events are generated frequently, this could in theory cause stutters.

I’ve recently noticed bad performance (stutters) in my applications, too, unless I disableFrameRate() and gl::enableVerticalSync( false ). So far I haven’t been able to 100% figure out what’s causing it.

Part of the update loop in Cinder was written by me (see here for a bit of history). I know Cinder’s frame rate limiter keeps processing events while waiting until it’s time to call update() and draw() again, so I don’t think there is something wrong with Cinder itself.

To summarize:

  • if the problem is gone when you disable vertical sync, you can blame the GPU driver for spin-waiting. If using NVIDIA hardware, use its control panel to either:
  1. disable Threaded Optimization, enable Vertical Sync, enable Triple Buffering
  2. or: enable Threaded Optimization, disable Vertical Sync, disable Triple Buffering
  • if the problem is gone when you disable the frame rate limiter, you can blame the Cinder update loop.
  • if nothing helps and you only experience a performance drop when you move the mouse, inspect your code to make absolutely sure it’s not doing anything on mouse moves.

-Paul

This very odd. I have tried what you suggested and i still get the stuttering when moving the mouse. It is not a performance thing for me, frame rate stays at 59-60 average fps.

This stuttering also happens when i press a key. And cant really notice it happening in Debug either.

I have now commented out all mouse and keyboard functions from my App class (i.e i am not overriding mouse up/down/move/drag and key up/down functions) and it still happens.

The app i am making is a soccer game and the stuttering is affecting the players and ball when they move. When they are static there is no stuttering. I had thought that somehow my camera tracking code was being affected by input from the mouse or keyboard but cannot see how this is possible, All input comes from my App class, no where am i connecting to a signal or message outside of the App class methods for input ().

I will keep looking. Thank you for the help.

You say that you have a steady average fps of 59-60. Is this with vertical sync and frame rate disabled? If so, you must be CPU limited and your CPU must be running at 100%. And in that case the operating system might do some processing whenever you move the mouse that eats a bit of processor time, causing a slight drop in fps. Can you confirm this?

-Paul

This is all i have in prepareSettings function

ci::gl::enableVerticalSync(true);

My cpu is running at 16% for my app, 19% overall.

I am thinking it might be a rendering issue that is only noticeable in release because of the higher framerate. If rendering is skipped for a few frames you notice it more because the game is much smoother to watch.

In debug my app gets 30ish average fps. If the rendering misses a few frames it is not as noticeable because the gameplay choppy anyway from the bigger delta time?

Got the stuttering to stop!!!

So i never tried switching threading optimisation off like you suggested. This was because disabling vertical sync didn’t make the stuttering stop.

if the problem is gone when you disable vertical sync, you can blame the GPU driver for spin-waiting.

I tried switching threading optimisation off and kept vertical sync enabled and that fixed it.
Do i need triple buffering enabled and to disable vertical sync if its fixed now? Will i get problems somewhere else if i dont?

Good to hear you found the problem! It’s been a while since I last researched it, but it seems NVIDIA’s driver is still causing this stuttering behavior. See for example this StackOverflow post. Apparently you can disable Threaded Optimization (TO) from your code.

I measured my current project’s performance with and without it and found that with TO disabled the experience is measurably smoother (~0.17ms per frame) but uses a little bit more CPU time (0.22ms per frame in my project). With TO enabled some frames take 16.6ms extra (that’s a whole frame!), but CPU is at a low 0.04ms per frame. I happily take the extra CPU cost if it means the experience is smoother.

-Paul

P.S.: you don’t have to enable triple buffering. It does not do much at 60 fps. It only helps at lower frame rates (<30fps). Disabling vertical sync will let your game run at the highest frame rate, but will also introduce tearing.

I usually disableFrameRate() and enable vertical sync. I also make my application frame rate independent. If you want your application to become frame rate independent without altering your code too much, let me know and I’ll post a bit of code I always use in my projects.

1 Like

It would be great if you could share any resources that can help us learn the concepts behind frame rate independence. The notes you share are quite instructive as well.

Thank you!
Bala.

I found my CPU is running at 1.6% (was 16%) for my app and 6%-9%(was 19%) overall after disabling TO.This is using the task manager to measure. So i have seen a decrease in CPU usage!

I need vertical sync as tearing is not acceptable for my apps use.

As for making my game framerate independent, i think it is already as i have this code…

m_lastElapsedSeconds = m_elapsedSeconds;
m_elapsedSeconds = ci::app::getElapsedSeconds();

float fDeltaFloat = (float)( m_elapsedSeconds - m_lastElapsedSeconds );
m_GameFSM.Update(fDeltaFloat);

Which means my logic code will be aware of the time that has passed since the last update.

Thank you for the help, Paul. I have been trying to figure this out for a while.

Hi,

what you have there is a way to measure elapsed time. This is sufficient if all your code is time-based. For example, if you want to move an object with a certain velocity, you could use good old physics like this:

position += elapsed_time * velocity;

But not all physics or simulations can easily be written like that. What if you have colliding objects? Or gravity? As you will probably know, a lot of simulations use simple Euler integration:

acceleration = force / mass;
velocity += acceleration;
position += velocity;

or sometimes Verlet integration:

auto velocity = current_position - previous_position;
previous_position = current_position;
current_position += velocity;
current_position += gravity;

Anyway, the point is that time is not used in these equations and it is expected that the simulation runs a fixed number of steps per second. While the above equations can be easily rewritten with a time component, it becomes way more complex when you start performing collision detection and response, or flocking, or magnetism, or rope physics. Anything where objects react to each other based on range. The above equations allow you to handle those situations easily, as long as they run, for example, 60 times a second.

Enter the following update loop:

void MyApp::update()
{
    // Use a fixed time step for a steady 60 updates per second.
    static const double timestep = 1.0 / 60.0;

    // Keep track of time.
    static double time = getElapsedSeconds();
    static double accumulator = 0.0;

    // Calculate elapsed time since last frame.
    double elapsed = getElapsedSeconds() - time;
    time += elapsed;

    // Update your simulation/physics/animations/objects.
    accumulator += math<double>::min( elapsed, 0.1 ); // prevents 'spiral of death'

    while( accumulator >= timestep ) {
        m_GameAppPS.Update( mIsPaused ? 0.0 : timestep );

        accumulator -= timestep;
    }
}

private:
    bool mIsPaused;

The above code will make sure that the m_GameAppPS.Update() method is always called exactly 60 times a second, regardless of the actual frame rate of your application. Or more precisely: it will be called 60 times per second on average. Sometimes it’s 59 times, sometimes 61, but measured over a long period of time it will be exactly 60 times per second.

If your frame rate is exactly 60 fps, update will be called once per draw (so the loop is similar to Cinder’s default loop: update-draw-update-draw). If your frame rate is 120, the update will be called every other frame (so the loop becomes: update-draw-draw). If it’s 30, the update will be called twice per frame (update-update-draw). If your frame rate drops to 6 fps, it will start skipping frames to prevent a so-called spiral of death. This should not happen in a well-tested application, of course, but will happen while debugging, so it’s nice that your application will be able to recover. You can test this yourself if you compile this Gist.

Why do I think this is better than using time? Because reasons. For example, Box2D or Bullet want you to call their update or simulate functions with a relatively steady time between calls. It results in smoother and more predictable physics. Your own equations become much simpler as well and you’re free to use whatever integration method you’d like. You never have to worry that your animations run too fast or too slow anymore.

Of course, you can still use time if you want to. My game objects all keep track of their own time. A double mTime member variable is enough. This allows you to independently slow down, speed up, restart or delay objects, while you can still pause and resume your application. The update method of my objects looks a bit like this:

void update( double elapsed = 0.0 )
{
    mTime += elapsed;

    // Do something time-based, or perform a simple integration step. 
    // There are no limits here.
    const int32_t kVelocityIterations = 3;
    const int32_t kPositionIterations = 8;
    mBox2dWorld->Step( elapsed, kVelocityIterations, kPositionIterations );

    if( mTime >= 60.0 )
        gotoNextScene();
}

private:
    double mTime;

Pausing your application is easy, too. Either stop calling the m_GameAppPs.Update() entirely, or (recommended) call it with an elapsed time of 0 seconds.

Slow-motion? No problem:

mTime += 0.2 * elapsed;

Want to restart the timer?

mTime = 0.0;

Want to add a delay?

void start( double delay )
{
    mTime = -delay;
    mIsRunning = true;
}

void update( double elapsed )
{
    if( mIsRunning )
        mTime += elapsed;

    double time = glm::max( 0.0, mTime );
}

private:
    double mTime;
    bool   mIsRunning;

Love using Cinder’s Timeline? With good reason. Just add your own timeline member variable and do this:

void update( double elapsed )
{
    mTimeline->step( elapsed );
}

void fadein( float duration )
{
    mTimeline->apply( &mAlpha, 1.0f, duration );
}

private:
    ci::TimelineRef mTimeline;
    ci::Anim<float> mAlpha;

As you can see, you have total flexibility and control over the animations and simulations in your application.

Sorry for the long post, but hopefully it comes in handy. Oh, by the way, the update loop was inspired by this great article that is definitely worth a read.

-Paul

4 Likes

Thank you so much for the detailed post!

Thanks for taking the time to explain, Paul.

@paul.houx do you have any idea why the article you linked to does the step below? Still trying to get my head round the explanation given.

    const double alpha = accumulator / dt;

    State state = currentState * alpha + 
    previousState * ( 1.0 - alpha );

    render( state );

I thought it would be something like below instead. As all the time for that update has not been consumed by the physics( or game code) you must interpolate forward in time by a ratio of the amount left in accumulator to get the state to render.

const double alpha = accumulator / dt;

State state = currentState + (currentState * alpha);

render( state );

Hi,

yes, what he does there is indeed interpolate between the last known state and a future state (assuming no sudden changes) to reduce any stutters. Note that he does not really change the state, he just draws with interpolated values.

While this does result in smoother animation, it also greatly complicates the code and isn’t more “correct” (edit: see below). Which is why I left it out.

Your code should be correct if you fix your interpolation.

-Paul


Edit: for this additional step to be correct, you must make sure to use the correct order of updating your values, which in this case would be:

position += velocity;
velocity += acceleration;
acceleration = force / mass;

as opposed to what you’d normally do:

acceleration = force / mass;
velocity += acceleration;
position += velocity;