Vision callback functions

A child script, or a customization script can include a vision callback function, when associated with a vision sensor. When present for a given vision sensor, then the system will call the callback function everytime a new image was acquired or applied, allowing the user to perform image processing. This is the case with following API functions: sim.handleVisionSensor, sim.checkVisionSensor, sim.checkVisionSensorEx, sim.setVisionSensorImage, and sim.setVisionSensorCharImage.

Some conditions apply as to the location of the vision callback function: normally, it should be inside of a customization script, or a non-threaded child script. However, if the triggering API function (e.g. sim.handleVisionSensor) is called from within of a threaded child script, then the vision callback function should also be located in the threaded child script. If a vision callback function is present in a non-threaded child script as well as in a customization script, both attached to the vision sensor, then the child script will be called first, and the customization script second.

Following represents an empty vision callback function:

function sysCall_vision(inData)
    -- We have:
    -- inData.handle : the handle of the vision sensor.
    -- inData.resolution : the x/y resolution of the vision sensor
    -- inData.clippingPlanes : the near and far clipping planes of the vision sensor
    -- inData.viewAngle : the view angle of the vision sensor (if in persp. proj. mode)
    -- inData.orthoSize : the ortho size of the vision sensor (if in orth. proj. mode)
    -- inData.perspectiveOperation : true if the sensor is in persp. proj. mode

    local outData={}
    outData.trigger=false -- true if the sensor should trigger
    outData.packedPackets={} -- a table of packed packets. Can be accessed via e.g. sim.readVisionSensor
    return outData
end

Image processing can be performed by using various API functions. The vision plugin exports a few very simple image processing functions. Many more image processing functions are supported via the image plugin (OpenCV wrapper).

Following represents a simple edge detection vision callback function, that triggers and returns a packet of data (based on the vision plugin functions):

function sysCall_vision(inData)
    simVision.sensorImgToWorkImg(inData.handle)
    simVision.edgeDetectionOnWorkImg(inData.handle,0.1)
    simVision.workImgToSensorImg(inData.handle)

    local outData={}
    outData.trigger=true
    local packetData={1.0,42.123,129.3}
    outData.packedPackets={sim.packFloatTable(packetData)}
    return outData
end

Following represents a vision callback function, that draws a circle onto the acquired image (based on the image plugin functions):

function sysCall_vision(inData)
    local imgHandle=simIM.readFromVisionSensor(inData.handle)
    local center={inData.resolution[1]/2,inData.resolution[2]/2}
    local radius=(inData.resolution[1]+inData.resolution[2])/8
    simIM.circle(imgHandle,center,radius,{255,0,255},4)
    simIM.writeToVisionSensor(imgHandle,inData.handle)
    simIM.destroy(imgHandle)
end

Vision callback functions

Recommended topics