How to detect mouse hovering over a point

Newbie question…

I am working on a space game and basically have a point cloud representing stars in a VboMesh:

    _mesh = gl::VboMesh::create(positions.size(), GL_POINTS, { layout });
    _mesh->bufferAttrib(geom::POSITION, positions.size() * sizeof(vec3),;   
    _mesh->bufferAttrib(geom::COLOR, colors.size() * sizeof(vec4),;
    _shader = gl::getStockShader(gl::ShaderDef().color());
    _batch = gl::Batch::create(_mesh, _shader);

I would like to assign an ID of some sort to each point and detect if the mouse is hovering over ( or near ) to the point and fetch it’s ID so that I can display some data about the selected star. What is the best way to go about this?


Few different ways to go about it. Given the amount of particles involved a straight linear search is probably not going to work (i’d try it first though, as performance intuitions are very often proven wrong by profiling), so failing that you could:

  1. Stuff all your points into some kind of acceleration structure like a Kd-tree / Octree and query it
  2. Assign a colour ID to each particle and write it to a separate colour attachment in a gl::Fbo and query that using mouse coordinates (possibly inaccurate due to the size of the particles)
  3. If you’re on windows (i.e later versions of OpenGL), you could pass your mouse coordinates as a uniform to your shader and do a screen space proximity check at render time and write the result to an SSBO (this could be done with an fbo in a pinch if on older gl (i.e macOS))
1 Like

Thanks, I do have them in a Kd-tree for nearest neighbor and range searches, I will investigate further.

I am running on Windows, option 3 looks interesting as well.

Just did a quick test using PCL’s search API and it’s totally fast enough for interactive usage and only took about 10 lines of code to implement :slight_smile:

I didn’t know about PCL, thanks!

Did you calculate your origin and direction using CameraPesp::generateRay(mouse, getWindowSize())? I think I am missing a piece. My scaling is off in the y direction (screen space). The further I get from the center of screen in the y direction the bigger the error is. In other words my mouse has to be higher than the actual point to detect it ( or lower than the actual point in below the center of the screen ). Note this is not an issue in the x direction.

ie. the blue circle is the mouse cursor position and the green circle is the selected point.

Yep, this is the entirety of my intersection function:

void PointSpriteApp::mouseMove ( MouseEvent event )
    Ray r = _camera.generateRay( event.getPos(), getWindowSize() );
    Eigen::Vector3f o { &r.getOrigin().x };
    Eigen::Vector3f d { &r.getDirection().x };
    Timer t{true};
    pcl::Indices indices;
    bool hit = _octree.getIntersectedVoxelIndices(o, d, indices) > 0;
    if ( hit )
        auto c = _camera.getEyePoint();
        std::sort( indices.begin(), indices.end(), [=] ( auto& ai, auto& bi )
            auto& a = _positions[ai];
            auto& b = _positions[bi];
            return glm::distance2(a, c) < glm::distance2(b, c);
        } );
        _index = indices.back();
        _geometry->getGlslProg()->uniform( "uIndex", _index );
        _index = -1;
    std::cout << "Searched " << _positions.size() << " points in " << (t.getSeconds() * 1000.0f) << "ms" << std::endl;

Are you on a high-dpi or scaled display by any chance? If so you might want to use app::toPixels ( getWindowSize() ) to compensate for the scale ratio. Just tested this, has no effect. Sorry.

1 Like

Wondering if this has to do with your tree resolution, do you have any way to visualise it? If the voxels are larger towards the extremities due to your stars being more sparse out there, it makes sense that the voxel center would deviate more from the actual point the further away from the middle you get.

Tonight after work I think I will drop the tree and try a brute force method to confirm everything else is good.

Found an interesting result, still investigating but if I use the alternate function for GenerateRay it seems to work correctly

    float u = _mouse.x / (float)getWindowWidth();
    float v = _mouse.y / (float)getWindowHeight();
    Ray ray =, 1.0f - v,;
    Ray ray2 =, _app->getWindowSize());

In other words ray works correctly but ray2 does not. Comparing the two there are small differences in the mDirection and mInvDirection vector between the two ray objects. Everything else appears to be equal.