The Dragonfly Java Application is a versatile system. It can be set in many different ways and is highly configurable. This page details the different options and settings this system can offer.

WARNING: before going further, please make sure that all the requirements stated in the previous support pages (Requirements and Setup) are met.

1 – Getting started

Here is a quick guide about how to get started with the Dragonfly Java Application. For a deeper knowledge about ALL the configuration options, please refer to the documentation available inside the Dragonfly Web UI under the tab Documentation.

1.1 – Launch

Do you have already a set of credentials? If not already done please generate a set of Accuware credentials (usernamepassword and Site ID) filling the form at this link.

The Dragonfly Java Application can be launched in different ways (whether you need a Web UI or not):

  • with the Dragonfly Web UI (that will be opened inside a new Firefox window):
    • OPTION 1 – run the application by clicking the Dragonfly icon that can be found within the Ubuntu Desktop Launcher menu shown in the image below.
      • Enter the credentials (username, password and Site ID) received with the Accuware Activation email. Once the credentials will be accepted by the Dragonfly Cloud server , the Dragonfly Java App will be launched and the Main tab will be available inside the Dragonfly Web UI.
    • OPTION 2 – open your browser and type the URL http://localhost:5000/. This can’t be done at first start and it can be done ONLY if the Dragonfly engine is already running.
  • without Dragonfly Web UI: go the directory where the start.sh script is located (cd dragonfly_linux/application/Dragonfly2Java_jar) and enter the following command:

About the internet connection: at first launch the credentials are immediately validated against the Accuware Dragonfly Cloud server. So an Internet connection is required! Validated credentials allow you to use the Dragonfly Java App for a certain amount of time (30 days) without the need for an Internet connection.

Troubles during the login? Please get in touch with us filling this form and attach the dragonfly2.log file available under the folder dragonfly_linux/application/Dragonfly2Java_jar/logs

Dragonfly launcher, coming with an installation on Ubuntu 16.04
Dragonfly launcher, coming with an installation on Ubuntu 16.04

1.2 – Configuration

A bit of configuration is required before starting to navigate. Please go to the Configuration tab to do so. The Configuration tab displays a lot of variables to be set. Most of them are not useful for the majority of users and DO NOT need to be touched.

The parameters that need to be changed before navigating are the following:

  • CAM_MODE – monocular or stereo, depending on what type of camera is used.
  • CAM_SOURCE – this is an important configuration parameter and has 2 variances:
    • The camera to consider is a USB camera: in this case you need to indicate the index of the camera to be used. The default value is -1, which advice the app to use the default system camera. If you have multiple cameras connected to your machine in order to identify the camera index, press the “List cameras” button on the right area of the tab. This will list all the cameras available, and provide the associated indexes. If 2 monocular cameras are used for stereoscopy, enter the 2 indexes separated by a coma, beginning with the index of the left camera.
    • The camera is a network camera: in this case you need to indicate the GStreamer pipelines or parameter strings to be used for by FFMPEG internally. Some ready to be used examples for GStreamer pipeline configurations and FFMPEG parameters are given inside this page.
  • CAM_CALIBRATION_FILE – this field should be filled with the full name of the JSON calibration file (without the path) that must be placed inside the config folder (cd dragonfly_linux/application/Dragonfly2Java_jar/config). How to generate the JSON calibration file?
    • Accuware Dragonfly Demonstration kit: if the Accuware Dragonfly Demonstration kit has been purchased the calibration of the camera has already been performed by the Accuware support team, and the JSON calibration file is already available inside the config folder of the Dragonfly Java Application on board of the mini-PC.
    • Accuware Online Calibration Tool: if you are using your own camera then you must generate the JSON calibration file using the Accuware Camera Calibration tool available at this link and place the calibration.json file generated by the tool inside the config folder.

Also, the user will probably be interested in the following parameters:

  • POSITION_UPLOAD_INTERVAL – If the value of this field is superior to 1000 ms, the device sends its position to the Accuware servers. The value defines the period at which the position is refreshed on the dashboard. An Internet connection is required. The real-time device location can be:
  • POSITION_UPLOAD_STORE – this enables to store the position in the Accuware servers, thus allowing to get access to the device location history. An Internet connection is required. The historical device locations can be:
  • POSITION_LOG_ENABLED  this enables to store the position history of the device locally in a CSV file. This file is located under the logs folder (cd dragonfly_linux/application/Dragonfly2Java_jar/logs), is named positionlog.cvs, contains every positions the system has computed and it has an automatic daily rollover mechanism built in. All the information about the CSV files can be found at this link.
  • VIDEO_RECORDING_ENABLED – this records a video of what the camera sees during the navigation. It can be used for monitoring or for debugging purposes. Please note that the video is recorded at the speed of the location system and not at a fixed standard frame rate, so if the system was running at about 14 FPS, the frame rate of the video is 14 FPS.

Once the parameters are set, press the Save Config and Restart button. The application should take between 3 and 5 seconds to restart and apply the changes.

Configuration tab

2 – Test the positioning

The Positioning consists of the two sub processes: Mapping and Localization (sometimes also called Navigation). i.e. exactly what the term SLAM means – simultaneous localization and mapping. When the settings are properly configured, the positioning can be started:

  • Go to the Main tab, and press the Start Positioning button. A preview of the camera should be displayed.
  • At start, the system state is MAP INIT. A slow translation motion (to the left or to the right) of the camera is required in order to initialize the system (try to do a translation and NOT a rotation!). The warm up does not take more than 3-4 seconds. We recommend to perform the initialization in a scene in which as many small objects as possible.
  • Once done the system state will change to NAVIGATION. At this point:
    • inside the camera preview blue marks attached to recognized features are shown inside the camera preview.
    • inside the plot the real-time location of your camera is shown as a moving drone! The flying drone is also reflecting the attitude of your device (pitch, roll and yaw).
  • The drone may already have a RED thread behind it. This RED thread is your track. While you are roaming, your position is tracked and the path is drawn in RED.
  • The color of the drone’s light changes depending on the status of the positioning mode:
    • GREEN – the Dragonfly engine can compute your location.
    • BLUE – the Dragonfly engine can NOT compute your location. In this situation, your device should also express a corresponding audible disappointment (“LOST”) and at the same time, all previously displayed blue feature marks should disappear. You can only correct this situation by slowly moving back to the position where you were last in the “NAVIGATION” state. Sometimes only a map reset helps to get back on track.
  • Use your mouse to zoom and rotate the plot if needed or to change the perspective on the scene.

Preview of the Main tab while navigating

NOTES:

  • Real horizon vs computed horizon: due to the fact that no world-reference is used (no floor plan is in use), the Dragonfly engine has no way to know exactly what is the real-world horizon. So the Dragonfly engine assumes that the floor is on the same horizontal axis as the horizontal axis of the camera during the MAP INIT phase. So, without world-references, if your camera is not perfectly horizontally held during the MAP INIT stage, there will be an angle between the absolute horizontal plane of the real-world and the computed horizontal plane of your device shown inside the plot.
  • Stereo mode: in stereo mode there is no such thing as map initialization and pure rotation limitation. The user can navigate without being limited in the way he moves. However, in stereo mode, the algorithm also runs slower than in monocular mode. It is thus advised to avoid moving too fast in stereo mode.
  • X,Y, Z vs WGS-84 coordinates: when using visual (or virtual) markers, when the 3rd visual (or virtual) marker is detected the Dragonfly Java App fetches the floor-plan (and optional geo-fences) from the Accuware server, and displays them. An Internet connection is required: the first time your floor plans/geo-fences are download from the Accuware cloud server and to download the most updated versions of your floor plans/geo-fences. In addition to this, the positions will be displayed with WGS-84 coordinates (latitude, longitude, altitude and level ID)More info about virtual and visual markers can be found here.
  • Dimension of the area to be mapped: we have customers who have mapped up to 40.000 sqm. Despite this we recommend to limit the area covered by a single map to 15.000 sqm. Anyway the constraint is the RAM and CPU available by the computing unit running the Dragonfly Java App (we are working on reducing these constraints).
  • Best practices about mapping: when first mapping the environment, it is recommend to make a big loop around the considered venue, and then to cross a couple of the main aisles if the venue is big. Then, loop-closing will be performed automatically and it is no longer required to specifically look for new loop-closings.

Info area

Below the camera preview are displayed some information:

  • Number of frame per seconds, providing indication about the speed at which the system runs.
  • Number of detected visual or virtual markersAdditional information here.
  • Number of map points, which provides an indication about the size of the map.
  • The current map loaded, which is set to undefined until the map is saved.
  • The position shown with un-dimensional values (vertical, lateral, horizontal) or as WGS84 coordinates (latitude, longitude, altitude) and level ID. Additional information here.
  • The attitude as pitch, yaw and roll. Additional information here.

3 – Maps management

When the 3 visual or virtual markers have been detected and the venue has been well discovered, the user may want to save the current state of exploration to avoid losing the work done. The Maps tab offers a couple of options to save the current map, load a different map, and also reset the map (which means restarting from scratch the positioning).

Save a map

In order to save the current map, fill the Save current map name and description, and press the Save button. If the saving has been successful, a new map will appear in the list below. This map can be loaded later, even after the Dragonfly Java Application has been closed.

Weight of the maps: the size of a map related to a 3D area of about 6000 m3 (made of about 20,000 Map Points) is about 100 MB.

Share a map between devices

The map management system implements a feature allowing to store the map files in the Accuware cloud server. Doing this, a map can be shared among all the devices registered on the same site. This avoids having to rediscover the whole area with every device. In order to save a map in the cloud, just press the Synchronize button on the right. An Internet connection is required. This will:

  1. upload on the Accuware cloud server:
    • all the locally saved maps that are more recent than the ones already existing on the Accuware cloud server.
    • all the NEW maps that exist locally but do not exist on the Accuware cloud server.
  2. download from the Accuware cloud server:
    1. all the maps detected as more recent than the local ones.
    2. all the NEW maps not available locally.

Load a map

To load a map, just select a map in the list and press the Load button. When a map is loaded, return to the Main tab. The name of the loaded map should be displayed in the information dashboard. If the navigation is started, it will automatically be resumed in the loaded map and the algorithm will try to recover its position from it.

NOTE: be advised that if a map is loaded, all the unsaved progress of the previous map will be lost.

Extend a map

If a map previously saved map has been reused and extended, it is possible to save the progress within the same map file by pressing the Update button.

Delete a map

If the user is willing to discard a saved map, it is possible to delete it by pressing the Delete button.

NOTE: be advised that when pressing this button, all the unsaved progress of the previous map will be lost.

Reset a map

If, for some reason, a map has been wrongly generated and the navigation experience is not satisfactory, it is possible to restart the algorithm from scratch without turning it off, by pressing the Reset map button. This will delete all the current progress, and restart the navigation with zero markers and zero map-points.

NOTE: be  advised that when pressing this button, all the unsaved progress of the previous map will be lost.

Just after saving the map
Just after saving the map

4 – (OPTIONAL) Latitude and longitude

To be able to see the locations provided by the Dragonfly engine as WGS-84 coordinates (latitude and longitude and altitude) you need to perform these 3 steps:

  • modify the Dragonfly Java App configuration so that it uses the WGS-84 coordinate system. Do it now! To do so:
    • Select the Configuration tab.
    • Select 3 Visual/virtual markers inside the MAP_CALIBRATION_METHOD parameter.
  • 4.1 – upload a detailed and scaled floor plan inside the Accuware dashboard. The floor plan is used as a reference system to set to setup the 3 visual (or virtual) markers.
  • 4.1 – perform the calibration of the map using 3 visual (or virtual) markers. The markers are used as ground-truth landmarks for the system and are required in order to project the device position onto the real-world with WGS-84 coordinates (latitude, longitude and altitude).

4.1 – Upload a floor plan

To upload your floor plan:

  1. Access the Accuware dashboard at this page using your credentials.
  2. Go under General > Floor Plans, and follow the instructions.
  3. Use the tools at the top of the map to better position and scale the floor plan (View, Zoom In/Zoom Out, OpacityAspect ratioRuler). DO NOT STRETCH IT, and KEEP THE PROPORTIONS. The inaccuracies of the floor plan will be directly reflected on the position computed by the system. If the floor plan is 50 cm inaccurate at a point, then the position will be 50 cm inaccurate at that point.
  4. Choose the level in which you want to upload the floor plan, and (optional) assign a name and a description to the floor plan, and click on Save.

At the end of the upload process the page will display the floor plan just uploaded in the table above the map.

An Internet connection is required the first time your floor plans are download rom the Accuware cloud server and to download the most updated version of your floor plans from the Accuware cloud server. 

4.2 – Geo-reference a floor plan

4.2.1 – Visual markers

Visual markers are QR-code like patterns which have to be printed and deployed through the venue. It is required to create at least 3 markers, to print and deploy them through the venue prior to be able to see the locations as WGS-84 coordinates. Please note that to ensure good accuracy, it is advised that the markers are located at the extremities of the venue. In other words the best results are achieved when the 3 markers encompass the required area to be mapped. In addition to this there is no need for more than 3 markers.

INFO: The setup of visual markers is NOT necessary if the system is planned to be used with virtual markers described in the next section.

Step 1 – Create the visual markers

  1. Inside the Accuware dashboard select Dragonfly > Marking.
  2. Use the buttons in the top-left corner to select Floor plan.
  3. Click on the floor plan to create a marker at the required location. It is mandatory to closely match the visual markers locations with the physical locations in which they will be placed. These are parameters you need to set:
    • Orientation: the orientation of the marker to the South-North axis. Use the slider’s arrow to set the orientation of your marker.
      1. If the marker lies on the floor, the orientation is the angle from the South-North direction to the direction the arrow displayed on the marker points to.
        Dragonfly - marker on floor
      2. If the marker is fixed on a wall, the orientation is the angle from the South-North direction to the direction the camera points to, if it looks orthogonal onto the marker. In this case, the arrow displayed on the marker has to point upwards.  
    • Altitude: the altitude of your marker to the floor in meters (e.g. 1.6 means 1.6 meters).
    • on Wall: this switch allows to tell if a marker is fixed on a wall (vertically fixed) or lies on the floor (horizontally fixed).
  4. Press Save to save the marker. Once saved, a green icon oriented according the settings chosen will be shown on the map.

You must create a total of 3 visual markers at different locations around your site by repeating steps 3 to 6.

Creation of a marker with the dashboard
Creation of a marker with the dashboard

Once the visual markers are created, it is required to print and deploy them in the real world exactly as they are set on the Dashboard. The real-world deployment must reflect the same position, altitude, orientation and disposition.

To print a marker:

  1. Go to Dragonfly > Markers using the side menu of the Accuware Dashboard.
  2. Click on a marker that has already been created.
  3. Click the Print button.

Once printed, place them where they belong inside the actual venue.

NOTE: please note that the markers are bound to their position and settings. Thus, if the position and/or settings of a marker have to be modified, it will be required to print it again.

Preview of a marker to be printed
Preview of a marker to be printed

4.2.2 – Virtual markers

If the use of visual marker is not an option, there is another way to perform the map calibration without using any printed markers. The setup of virtual markers requires to manually enter the current position of the camera on a floor plan at 3 different locations. The locations provided to the Dragonfly Java App while performing this calibration steps described below (clicking on the floor plan) are called Virtual markers (as opposed to the Visual markers described previously). An Internet connection is required the first time your floor plans are download rom the Accuware cloud server and to download the most updated version of your floor plans from the Accuware cloud server. 

NOTE: The setup of virtual markers is NOT necessary if the system is planned to be used with visual markers described in the previous section.

The actual process to define 3 virtual markers is the following:

  1. Start Dragonfly and Start positioning.
  2. Once the system is in the Mapping state, go with the device at a location of your venue that you can easily identify on the floor plan.
  3. When the location is reached, go to the Map calibration tab at the top of the positioning window, and select the level corresponding to the current level.
  4. Click on the floor plan at the position the device currently is. A pop-up asks for the altitude value. Enter the real altitude value and click Save marker.
  5. While still being in the Mapping state, go to a second different remarkable location with the device, then repeat 3 and 4.
  6. While still being in the Mapping state, go to a third different remarkable location with the device, then repeat 3 and 4.
  7. After the third location is entered, the system should switch to Navigation state and be able to provide the accurate position of the device as WGS-84 coordinates (+ altitude).

Calibration with virtual markers
Calibration with virtual markers

5 – Quit

In order to quit the Dragonfly Java Application, enter the Configuration tab and press the Terminate button. It is indeed not sufficient to close the browser, as closing the browser just turns off the UI Web application, but not the Dragonfly server.

6 – Hard reset

If, for some reasons, the Dragonfly Java Application does not work as expect, or refuses to start even after a machine reboot, it is possible to do a hard reset of the system and to restart it from scratch. To do so:

  1. Kill any running instance of Dragonfly Java App by pressing the Terminate button inside the Configuration tab.
  2. Go to the root folder of the Dragonfly installation (cd dragonfly_linux/application/Dragonfly2Java_jar)
  3. Delete the jsmetamaps and logs folders.
  4. Go open the config folder and delete ALL the .atkn files and the dragonfly2.properties file.

When the Dragonfly Java App will be restarted, it will be as like a brand new first start.

About the maps: note that the local maps are going to be deleted as well, but not the maps on the cloud. In order to keep the local maps, do not delete the maps folder.

Folders to be deleted at the root of Dragonfly installation
Folders to be deleted at the root of Dragonfly installation

Files to be deleted in the config folder
Files to be deleted in the config folder

NEXT STEP – How to update the Dragonfly Java App