Continues training
Drones to A.I to Drones
With the rise of drone usage, it’s easier to take aerial footage than ever before. The resulting data can trigger quick, effective action; removing guesswork and increasing aerial awareness, which can have profound implications on growing profits and trimming expenses.
And as drone use rises, so does the usage of AI, to navigate, detect, identify, and track meaningful artifacts and objects.
Some companies empower drones to collect hi-res photographic imagery. Then they bring it back for analysis and offline decision-making. (i.e, optimizing crop output or spotting flaws at a construction site); others focus on real-time inference, tracking and alerts (i.e improving situational-awareness for security and military teams). Sightx seeks to combine these two functions to drones to become a semi-autonomous platform that, while doing its “real job” of aerial reconnaissance with AI models, gathers data that will later enhance the same AI models – perpetuating a cycle of learning and improvement.
In short, Sightx pursues the Holy Grail of automation. Following each return to base to offload data, a drone can return to the sky with an upgraded model. The previous trip has retrained the drone for even better detection and tracking than on its previous flight – all with minimal or zero human intervention.
Although the models originally focused on land terrain, Sightx is working on processing air and sea footage. The researchers and developers are required to address various modalities of wavelengths, such as RGB, SWIR, and MWIR. Each terrain presents specific challenges and requires an arsenal of technical abilities.
MLOps-First Approach
To fully realize this vision, Sightx takes an MLOps-first approach, with a streamlined pipeline designed and optimized to minimize (or ideally eliminate) human interaction; an appearance of a new data set should exclusively start the process. With so much reliance on MLOps, Sightx’s developers knew they had to deploy only best-of-breed products and practices in building this pipeline.
How does it work:
- A drone, enhanced with a Sightx AI model connected to its camera, hovers over the targeted area, detects and photographs a collection of objects, and returns to base.
- The images and the corresponding flight data and parameters are then classified. The most valuable and representative are sent for manual annotation to identify object characteristics.
- Once annotation is complete, data is fed back into the data management system, which triggers a retraining and validation process. Assuming the model has improved, it is deployed to the drone for another flight, where it can more accurately perform its mission.
Using your AI model in real life
The Computer Vision model Sightx is building sounds like a classic, straight-forward AI implementation. But as industry experts know, well-trained models are just one part of a high-performing AI system; Sightx’s challenge is even more complex as their software is designed specifically for edge devices. The C++ code and ML models need to be compiled and optimized to run on various platforms. From Intel’s Tiger Lake mobile processor and Nvidia’s NX to many other ARM-based SOMs with GPUs and DSPs.
Orchestration and experiment management as a solution for deploying different AI models and codes
Sightx deploys ClearML’s Orchestrator as the core of its automation. ClearML’s Experiment Manager is used to log and produce experiments and ClearML-Agent to orchestrate machines for various model training tasks. With ClearML, Sightx constructs an AirFlow and S3-powered ETL infrastructure that enables a drone to upload new data as fast as possible, for rapid retraining and next-day performance improvements in the air. With a planned integration of ClearML’s platform and Kibana’s, Sightx is planning to create dashboards with clear data visualizations that allow for engineers, researchers, and decision-makers to keep track of the training process.
The Sightx research team utilizes ClearML’s powerful tools to build Computer Vision models that can detect, track, and classify objects; they are also working on cutting-edge abilities like re-identifying objects that were previously seen by the system and were saved in the internal memory of the drone itself. . It can conduct experiments with advanced model optimization techniques such as quantization, pruning, and knowledge distillation.
The DevOps team worked hard on building this well-orchestrated cloud-based and docker-based infrastructure to enable fast and stable CI/CD workflow. The process has been so successful that in addition to marketing the models, Sightx began to sell the entire end-to-end training pipeline as a stand-alone service/product for companies looking for similar results.
Conclusion
This technology really pushes a new paradigm– empowering the drone to be the catalyst to its own continuous improvement process. ClearML’s suite of tools made it easier to develop virtually every stage of the process to create a successful product. It is easier focusing on the core innovation and not the labor-intensive CI/CD workflow. These processes could otherwise distract and delay time-to-market.
