The Dragonfly Java Application is a very powerful positioning and navigation system that can be customized in different ways. This page details all options and settings this system can offer and how to use it properly to compute the location of your devices.

WARNING: before going further, please make sure that all the requirements stated in the previous support pages (Requirements and Setup) are met.

1 – Getting started

Here is a quick guide about how to get started with the Dragonfly Java Application. For a deeper knowledge about ALL the configuration options, please refer to the documentation available inside the Dragonfly Web UI under the Documentation tab.

1.1 – Launch

Do you have already a set of credentials? If not already done please generate a set of Accuware credentials (usernamepassword and Site ID) filling the form at this link.

The Dragonfly Java Application can be launched in different ways (whether you need the Dragonfly Web UI or not):

  • with the Dragonfly Web UI:
    • OPTION 1 – run the application by clicking the Dragonfly icon that can be found within the Ubuntu Desktop Launcher menu (as shown in the image below). Enter the credentials (username, password and Site ID) received with the Accuware Activation email. Once the credentials will be accepted by the Dragonfly Cloud server, the Dragonfly Java App will be launched and the Main tab will be available inside the Dragonfly Web UI.
    • OPTION 2 – open your browser and type the URL http://localhost:5000/. This can’t be done at first start and it can be done ONLY if the Dragonfly engine is already running.
  • without Dragonfly Web UI: go the directory where the start.sh script is located (cd dragonfly_linux/application/) and enter the following command:

About the internet connection: at first launch the credentials are immediately validated against the Accuware Dragonfly Cloud server (so an Internet connection is required). Validated credentials allow you to use the Dragonfly Java App for a certain amount of time (30 days) without the need for an Internet connection.

Troubles during the login? Please get in touch with us filling this form and attach the dragonfly2.log file available under the folder cd dragonfly_linux/application/data/logs .

Dragonfly launcher, coming with an installation on Ubuntu 16.04
Dragonfly launcher, coming with an installation on Ubuntu 16.04

1.2 – Configuration

A bit of configuration is required before starting to navigate. Please go to the Configuration tab to do so. The Configuration tab displays several variables grouped into 2 groups: General Settings and Advanced Settings.

These are the General Settings parameters that need to be changed before starting the positioning:

  • CAM_MODE – monocular or stereo, depending on what type of camera is used.
  • CAM_SOURCE – this is an important configuration parameter and has 2 variances:
    • if you are using 1 USB monocular camera directly connected to the Dragonfly machine you need to indicate the index of the camera to be used  (press the List cameras button to get the index or leave it to -1 to use the default system camera).
    • if you are using 1 USB stereoscopic camera directly connected to the Dragonfly machine you need to indicate the index of the camera to be used (press the List cameras button to get the index). There is a common bug on Ubuntu 18.04 where 2 indexes are displayed for single USB stereoscopic camera. Please use ONLY the first index displayed.
    • if you are using a stereoscopic camera (made of 2 USB monocular cameras directly connected to the Dragonfly machine) you need to indicate the indexes of the cameras to be used (press the List cameras button to get the indexes). Enter the 2 indexes separated by a coma, beginning with the index of the left camera.
    • if the camera is a network camera:
  • CAM_CALIBRATION_FILE – this field should be filled with the full name of the JSON calibration file (WITHOUT the path but WITH the extension .json) that should be placed inside the config folder (cd dragonfly_linux/application/data/config). It is potentially possible to start the positioning without a camera calibration file but it is not advised. The creation method of the calibration file depends on the type of camera used:
    • USB camera directly connected to the computing unit:
      • CASE 1 – if you have purchased the Accuware Dragonfly Demonstration kit (described below) the calibration of the camera has been performed already by the Accuware support team, and the JSON calibration file is already available inside the config folder of the Dragonfly Java Application on board of the mini-PC.
      • CASE 2 – if you are using a monocular camera connected via USB to the Dragonfly machine (included the ELP-USBFHD01M-L180) then you must generate the JSON calibration file using the Accuware Camera Calibration tool available at this link and place the JSON calibration file generated by the tool inside the config folder of the Dragonfly Java Application.
      • CASE 3 – if you are using the ASTAR stereoscopic camera connected via USB to the Dragonfly machine then:
        1. put the following URL inside your browser and replace the string {SERIAL_NUMBER} with the serial number written on the ASTAR stero cam.
          • https://cvnav.accuware.com/api/v1/dragonfly/calibration/{SERIAL_NUMBER}.json
        2. once done you will receive back a json file that must be placed inside the config folder of the Dragonfly Java Application.
      • CASE 4 – if you are using your own stereoscopic camera please contact us and let us know the exact model purchased.
    • Network camera that makes available its video stream to the Dragonfly machine using an Wi-Fi or mobile connection:
  • CAM_FOVwhen using:
    • the ELP MONOCULAR 180 degree camera please set this field to 120 degrees.
    • the ASTAR STEREO camera please set this field to 70 degrees.

IMPORTANT: once the parameters are set, press the Save Config and Restart button. The application should take between 3 and 5 seconds to restart and apply the changes.

General Settings panel

These are other parameters that might be of interest and available under the Advanced Settings area:

  • POSITION_UPLOAD_STORE – this enables to store the position in the Accuware Cloud server, thus allowing you to get access to the device location history. An Internet connection is required. The historical device locations can be:
  • POSITION_UPLOAD_INTERVAL – Number of milliseconds between two consecutive uploads of positions to the Accuware Dashboard. The value of this field MUST be superior to 1000 ms. An Internet connection is required. The real-time device location can be:
  • POSITION_LOG_ENABLED  this enables to store the position history of the device locally in a CSV file. This file is located under cd dragonfly_linux/application/data/logs, is named positionlog.cvs, contains every positions the Dragonfly engine has computed and it has an automatic daily rollover mechanism built in. All the information about the CSV files can be found at this link.
  • VIDEO_RECORDING_ENABLED – this records a video of what the camera sees during the navigation. It can be used for monitoring or for debugging purposes. Please note that the video is recorded at the speed of the location system and not at a fixed standard frame rate, so if the system was running at about 14 FPS, the frame rate of the video is 14 FPS. Videos are saved inside the directory cd dragonfly_linux/application/data/recorded and grouped by siteId.

IMPORTANT: once the parameters are set, press the Save Config and Restart button. The application should take between 3 and 5 seconds to restart and apply the changes.

Advanced Settings panel

2 – WGS 84 (latitude and longitude) and metric coordinates

To be able to see the locations provided by the Dragonfly engine as WGS 84 coordinates (latitude and longitude) or as metric coordinates (relative distances in meters from a specific origin [0,0]) you need to make sure the Dragonfly Java App is set properly to allow the calibration of the map. To do so:

  1. Select the Configuration tab.
  2. Make sure the MAP_CALIBRATION_METHOD parameter is set to Three markers used (either printed or virtual).

Once done you have to:

  • Section 2.1 – upload a detailed and scaled floor plan inside the Accuware dashboard. The floor plan is used as a reference system to setup the 3 visual (or virtual) markers.
  • Section 2.2 – perform the calibration of the map using 3 visual (or virtual) markers. The markers help in determining the relation of the internally used virtual coordinates to the coordinate system of the “real” world.

2.1 – Upload a floor plan

To upload your floor plan:

  1. Access the Accuware dashboard at this page using your credentials.
  2. From the TOP menu select General > Floor Plans > Create new floorplan.
    1. Choose the address.
    2. Choose a high quality floor plan image of your production/test environment with a maximum size of 5 MB each one. If you have a PDF floor plan, then you should convert it into an image file using Adobe Acrobat or using an online service like these.
    3. Select the Level ID, the Name and the Description. If you can’t select any of the listed levels, it means that there is already a floor plan image on each one of the levels and you need to create a NEW level!
    4. Move the floor plan image in its location. To help you with this process you can change the floor plan opacity, switch to the Satellite view and use the ruler tool available on the right side of the map. Make sure to scale the floor plan properly, DO NOT STRETCH IT, and KEEP THE PROPORTIONS. The inaccuracies of the floor plan will be directly reflected on the position computed by the system.
  3. Click on DONE and wait for the floor plan to be saved.

An Internet connection is required only:

  • the first time your floor plans are downloaded from the Accuware cloud server.
  • (if needed) to download the most updated version of your floor plans from the Accuware cloud server.

2.2 – Calibration of the map

2.2.1 – Virtual markers

The setup of virtual markers requires to manually enter the current position of the camera on a floor plan at 3 different locations. The locations provided to the Dragonfly Java App while performing this calibration steps described below (clicking on the floor plan) are called virtual markers (as opposed to the visual markers described in the next section).

Note: the setup of virtual markers is NOT necessary if the system is planned to be used with visual markers described in the next section.

The actual process to define 3 virtual markers is the following:

  1. Start Dragonfly and Start positioning. If you are using a network camera, please make sure you have started the stream on the remote device (Android/iOS/Raspberry Pi)
  2. At start, the system state is MAP INIT. A slow translation motion (to the left or to the right) of the camera is required in order to initialize the system (please perform a translation and NOT a pure rotation!). The warm up does not take more than 3 seconds. We recommend to perform the initialization in a scene in which there are as many small objects as possible.
  3. Once done the system state will change to MAPPING state.
  4. Move with the device towards the first remarkable location that will represent your first VIRTUAL MARKER.
  5. When the location is reached, select the Map calibration tab at the top of the positioning window, and select the level corresponding to the current level.
  6. Click on the floor plan at the position the device currently is. A pop-up asks for the altitude value. Enter the real altitude value and click Save marker.
  7. While still being in the MAPPING state, go to a second different remarkable location with the device, then repeat steps 5 to 6.
  8. While still being in the MAPPING state, go to a third different remarkable location with the device, then repeat steps 5 to 6.
  9. After the third location is entered, the system should switch to NAVIGATION state and be able to provide the accurate position of the device as:
    1. WGS 84 coordinates (latitude, longitude) + altitude (in meters).
    2. metric coordinates (relative distances in meters from a specific origin [0,0]) + altitude (in meters).
  10. Please click on the Position tab to see your accurate location as a BLUE dot on the floor plan!
  11. Colored dots attached to recognized features are shown inside the camera preview. The colors (and sizes) of the dots corresponds to the distance of the features to the camera. Big blue dots are close while small red dots are far away. This helps to detect when the computed map is wrong.

PLEASE READ: remember that if you are using a MONOCULAR camera the system is able to compute the location when:

  • the camera is pointed towards areas already MAPPED.
  • OR when the camera is moved (with translation motions) from a MAPPED area towards an UNMAPPED area (rich of features).

Blue dot shown after the calibration of the map using the virtual markers

Info area

Below the camera preview there are several interesting information:

  • Number of frame per seconds, providing indication about the speed at which the system runs.
  • Number of detected visual or virtual markers.
  • Number of 3D map points, which provides an indication about the size of the map. The 3D map points (a.k.a. 3D points cloud) can be visualized using the button available inside the plot when the Dragonfly Java App is used in Direct view mode.
  • The current map loaded, which is set to undefined until the map is saved.
  • The position shown as WGS 84 coordinates (latitude, longitude) + altitude (in meters).
  • The level ID.
  • The position shown as metric coordinates (relative distances in meters to a specific origin [0,0]) + altitude (in meters). By default the origin is the center of the floor plan:
    • X: positive when moving to the right. Negative when moving to the left.
    • Y: positive when moving to the top. Negative when moving to the bottom.
  • The attitude as pitch, yaw and roll. Additional information here.

Do you have questions? most of the answers you are looking for can be found inside this page.

Problems or bad results? please contact us, describe your problem and send us:

  • the exact version of your OS (macOS/Linux).
  • the exact model of the camera used.
  • the JSON calibration file of your camera – you can find it under the folder dragonfly_linux/application/data/config
  • the Dragonfly LOG file (dragonfly2.log) – you can find it under the folder dragonfly_linux/application/data/logs
  • the Dragonfly PROPERTIES file (dragonfly2.properties) – you can find it under the folder dragonfly_linux/application/data/config
  • the video recorded by the camera during the navigation after having removed the CAM_FOV parameter and enabled the VIDEO_RECORDING_ENABLED parameter inside the Configuration tab of the Dragonfly Web UI. You can find the video under the folder dragonfly_linux/application/data/recorded  Please remember to click on Stop Navigation at the end of the navigation to save the video!
  • a recording of the screen of the Dragonfly WEB UI during the navigation. This can be done using one of the many screen recorders available online for free (e.g. Kazam for Linux).

2.2.2 – Visual markers

Visual markers are QR-code like patterns which have to be printed and deployed through the area. It is required to create at least 3 markers, to print and deploy them through the venue prior to be able to see the locations as WGS 84 coordinates. Please note that to ensure good accuracy, it is advised that the markers are located at the extremities of the venue. In other words the best results are achieved when the 3 markers encompass the required area to be mapped. In addition to this there is no need for more than 3 markers.

Note: if you can’t attach physical visual markers inside your environment (e.g. a museum), don’t worry. You can use virtual markers described in the previous section.

Step 1 – Create the visual markers

  1. Access the Accuware dashboard at this page using your credentials.
  2. From the TOP menu select Dragonfly > Marking.
  3. From the TOP right menu select the Level of interest. 
  4. Click on the floor plan to create a marker at the required location. It is mandatory to closely match the visual markers locations with the physical locations in which they will be placed. These are parameters you need to set:
    • Orientation: the orientation of the marker to the South-North axis.
      1. If the marker lies on the floor, the orientation is the angle from the South-North direction to the direction the arrow displayed on the marker points to.
        Dragonfly - marker on floor
      2. If the marker is fixed on a wall, the orientation is the angle from the South-North direction to the direction the camera points to, if it looks orthogonal onto the marker. In this case, the arrow displayed on the marker has to point upwards.  
    • Altitude: the altitude of your marker to the floor in meters (e.g. 1.6 means 1.6 meters).
    • on Wall: this switch allows to tell if a marker is fixed on a wall (vertically fixed) or lies on the floor (horizontally fixed).
  5. Optionally you can assign a Label to the marker.
  6. Press Save to save the marker. Once saved, a green icon oriented according the settings chosen will be shown on the map.

You must create a total of 3 visual markers at different locations around your site by repeating steps 3 to 6.


Creation of a marker with the dashboard

Once the visual markers are created, it is required to print and deploy them in the real world exactly as they are set on the Dashboard. The real-world deployment must reflect the same position, altitude, orientation and disposition.

To print a marker:

  1. Go to Dragonfly > Markers using the side menu of the Accuware Dashboard.
  2. Click on a marker that has already been created.
  3. Click on QR code and then Print.

Once printed, place them where they belong inside the actual venue.

Note: please note that the visual markers are bound to their position and settings. Thus, if the position and/or settings of a visual marker have to be modified, it will be required to print it again.


Preview of a marker to be printed

Step 3 – Calibration using the visual markers

After the third visual marker has been placed you can proceed with the calibration of the map. The actual process to calibrate the map is the following:

  1. Start Dragonfly and Start positioning. If you are using a network camera, please make sure you have started the stream on the remote device (Android/iOS/Raspberry Pi).
  2. At start, the system state is MAP INIT. A slow translation motion (to the left or to the right) of the camera is required in order to initialize the system (please perform a translation and NOT a pure rotation!). The warm up does not take more than 3 seconds. We recommend to perform the initialization in a scene in which there are as many small objects as possible.
  3. Once the system is in the MAPPING state, go with the device to the location of your venue where there is the first visual marker. The Markers value inside the info area will increase of 1 unit.
  4. While in the MAPPING state, go with the device to the location of your venue where there is the second visual marker. The Markers value inside the info area will increase of 1 unit.
  5. While still being in the MAPPING state, go with the device to the location of your venue where there is the third visual marker. The Markers value inside the info area will increase of 1 unit.
  6. After that the third marker has been detected the system should switch automatically to NAVIGATION state and be able to provide the accurate position of the device as:
    • WGS 84 coordinates (latitude, longitude) + altitude (in meters).
    • metric coordinates (relative distances in meters from a specific origin [0,0]) + altitude (in meters).
  7. Please click on the Position tab to see your accurate location as a BLUE dot on the floor plan!
  8. Colored dots attached to recognized features are shown inside the camera preview. The colors (and sizes) of the dots corresponds to the distance of the features to the camera. Big blue dots are close while small red dots are far away. This helps to detect when the computed map is wrong.

PLEASE READ: remember that if you are using a MONOCULAR camera the system is able to compute the location when:

  • the camera is pointed towards areas already MAPPED.
  • OR when the camera is moved (with translation motions) from a MAPPED area towards an UNMAPPED area (rich of features).

Blue dot shown after the calibration of the map using the visual markers

Info area

Below the camera preview there are several interesting information:

  • Number of frame per seconds, providing indication about the speed at which the system runs.
  • Number of detected visual or virtual markers.
  • Number of 3D map points, which provides an indication about the size of the map. The 3D map points (a.k.a. 3D points cloud) can be visualized using the button available inside the plot when the Dragonfly Java App is used in Direct view mode.
  • The current map loaded, which is set to undefined until the map is saved.
  • The position shown as WGS 84 coordinates (latitude, longitude) + altitude.
  • The level ID.
  • The position shown as metric coordinates (relative distances in meters to a specific origin [0,0]). By default the origin is the center of the floor plan:
    • X: positive when moving to the right. Negative when moving to the left.
    • Y: positive when moving to the top. Negative when moving to the bottom.
  • The attitude as pitch, yaw and roll. Additional information here.

Do you have questions? most of the answers you are looking for can be found inside this page.

Problems or bad results? please contact us, describe your problem and send us:

  • the exact version of your OS (macOS/Linux).
  • the exact model of the camera used.
  • the JSON calibration file of your camera – you can find it under the folder dragonfly_linux/application/data/config
  • the Dragonfly LOG file (dragonfly2.log) – you can find it under the folder dragonfly_linux/application/data/logs
  • the Dragonfly PROPERTIES file (dragonfly2.properties) – you can find it under the folder dragonfly_linux/application/data/config
  • the video recorded by the camera during the navigation after having removed the CAM_FOV parameter and enabled the VIDEO_RECORDING_ENABLED parameter inside the Configuration tab of the Dragonfly WEB UI. You can find the video under the folder dragonfly_linux/application/data/recorded  Please remember to click on Stop Navigation at the end of the navigation to save the video!
  • a recording of the screen of the Dragonfly WEB UI during the navigation. This can be done using one of the many screen recorders available online for free (e.g. Kazam for Linux).

3 – Un-dimensioned raw XYZ coordinates

Preview of the Main tab while navigating

The Direct View mode allows the visualization of the position of your device inside a 3D plot as a drone moving in an un-dimensioned plot with raw XYZ coordinates. To test the Direct View mode:

  1. Select the Configuration tab.
  2. Inside the MAP_CALIBRATION_METHOD field select No markers used (either printed or visual).
  3. Click on Save Config and Restart.
  4. Go to the Main tab, and press the Start Positioning button. A preview of the camera should be displayed. If you are using a network camera, please make sure you have started the stream on the remote device (Android/iOS/Raspberry Pi)
  5. At start, the system state is MAP INIT. A slow translation motion (to the left or to the right) of the camera is required in order to initialize the system (please perform a translation and NOT a pure rotation!). The warm up does not take more than 3 seconds. We recommend to perform the initialization in a scene in which there are as many small objects as possible.
  6. Once done the system state will change to NAVIGATION. At this point:
    1. inside the camera preview blue marks attached to recognized features are shown inside the camera preview.
    2. inside the plot the real-time location of your camera is shown as a moving drone! The flying drone is also reflecting the attitude of your device (pitch, roll and yaw).
  7. The drone may already have a RED thread behind it (this is your track).
  8. The drone has a light on its top. The color of the drone’s light changes depending on the status of the positioning mode:
    • GREEN – the Dragonfly engine can compute your location.
    • BLUE – the Dragonfly engine can NOT compute your location. In this situation, the Dragonfly Java App also express a corresponding audible LOST message and at the same time, all previously displayed blue feature marks should disappear. You can only correct this situation by slowly moving back to the position where you were last in the “NAVIGATION” state.
  9. Use your mouse to zoom and rotate the plot if needed or to change the perspective on the scene.
Info area

Below the camera preview there are several interesting information:

  • Number of frame per seconds, providing indication about the speed at which the system runs.
  • Number of detected visual or virtual markers.
  • Number of 3D map points, which provides an indication about the size of the map. The 3D map points (a.k.a. 3D points cloud) can be visualized using the button available inside the plot when the Dragonfly Java App is used in Direct view mode.
  • The current map loaded, which is set to undefined until the map is saved.
  • The position shown as WGS 84 coordinates (latitude, longitude) + altitude (in meters).
  • The level ID.
  • The position shown as metric coordinates (relative distances in meters to a specific origin [0,0]) + altitude (in meters). By default the origin is the center of the floor plan:
    • X: positive when moving to the right. Negative when moving to the left.
    • Y: positive when moving to the top. Negative when moving to the bottom.
  • The attitude as pitch, yaw and roll. Additional information here.

4 – Maps management

When the 3 visual or virtual markers have been detected and the venue has been well discovered, the user may want to save the current state of exploration to avoid losing the work done. The Maps tab offers a couple of options to save the current map, load a different map, and also reset the map (which means restarting from scratch the positioning).

Save

In order to save the current map, fill the Save current map name and description, and press the Save button. If the saving has been successful, a new map will appear in the list below. This map can be loaded later, even after the Dragonfly Java Application has been closed.

Weight of the maps: the size of a map related to a 3D area of about 6000 m3 (made of about 20,000 Map Points) is about 100 MB.

Share

The map management system implements a feature allowing to store the map files in the Accuware Cloud server. Doing this, a map created by one device can be shared among all the other devices running the Dragonfly Java App and registered on the same site. This avoids having to re-map the whole area with every new device that will make use of the Dragonfly Java App. In order to save a map in the cloud, just press the Synchronize button on the right. An Internet connection is required. This will:

  1. upload on the Accuware cloud server:
    • all the locally saved maps that are more recent than the ones already existing on the Accuware cloud server.
    • all the NEW maps that exist locally but do not exist on the Accuware cloud server.
  2. download from the Accuware cloud server:
    1. all the maps detected as more recent than the local ones.
    2. all the NEW maps not available locally.

Load

To load a map, just select a map in the list and press the Load button. When a map is loaded, return to the Main tab. The name of the loaded map should be displayed in the information dashboard. If the navigation is started, it will automatically be resumed in the loaded map and the algorithm will try to recover its position from it.

NOTE: be advised that if a map is loaded, all the unsaved progress of the previous map will be lost.

Extend

If a map previously saved map has been reused and extended, it is possible to save the progress within the same map file by pressing the Update button.

Delete

If the user is willing to discard a saved map, it is possible to delete it by pressing the Delete button.

NOTE: be advised that when pressing this button, all the unsaved progress of the previous map will be lost.

Reset

If, for some reason, a map has been wrongly generated and the navigation experience is not satisfactory, it is possible to restart the algorithm from scratch without turning it off, by pressing the Reset map button. This will delete all the current progress, and restart the navigation with zero markers and zero map-points.

NOTE: be  advised that when pressing this button, all the unsaved progress of the previous map will be lost.

Just after saving the map
Just after saving the map

5 – Quit

In order to quit the Dragonfly Java Application, enter the Configuration tab and press the Terminate button. It is indeed not sufficient to close the browser, as closing the browser just turns off the Dragonfly Web UI application, but not the Dragonfly server.

6 – Hard reset

If, for some reasons, the Dragonfly Java Application does not work as expect, or refuses to start even after a machine reboot, it is possible to do a hard reset of the system and to restart it from scratch. To do so:

  1. Kill any running instance of Dragonfly Java App by pressing the Terminate button inside the Configuration tab.
  2. Go to the root folder of the Dragonfly installation (cd dragonfly_linux/application/data):
    • Delete the jsmetamaps and logs folders.
  3. Open the config folder (cd dragonfly_linux/application/data/config)
    • Delete ALL the .atkn files and the dragonfly2.properties files.

When the Dragonfly Java App will be restarted, it will be as like a brand new first start.

About the maps: note that the local maps are going to be deleted as well, but not the maps on the cloud. In order to keep the local maps, do not delete the cd dragonfly_linux/application/maps folder.

<< PREVIOUS