teknofest - fighting uavs 2021

introduction

as of 2021, it was decided to join teknofest fighting uav rotary wing competition as a team. here is personal experience i have gained throughout it.

before moving on, you can find the target of the competition and the details below, which were supposed to be achieved:

  1. custom-built drone from scratch with all the hardware included
  2. custom-trained computer vision model to process images
  3. ground station setup
  4. custom-coded gui for the competition to show each data shared by the telemetry
  5. all the possible safety checks
  6. judge server and local ground station integration and flawless communication for the data and video transmission and exchange

p.s all of the specifications can be found by clicking here

target:

by successfully passing all the requirements mentioned above, you are granted access to join the actual competition, which consists of 7 identically ruled parts. there was no team recorded, which could successfully pass all the failsafe checks and also actually detect and follow an opponent yet.

the award table is as following:

place fixed wing rotary wing
1st 250k try 50k try
2nd 150k try 40k try
3rd 100k try 30k try

used tech

  1. computer vision

    as it requests in the challenge, you are supposed to detect an opponent (any) and after the first detection you have to trigger the tracker, which is supposed to track and follow that opponent for the next 10 seconds straight. if this task is completed successfully, then you are earning 600 points alongside guaranteeing yourself to be in the top 3 teams. in order to solve this challenge, yolo v4 was used. while training a custom model on dozens of thousands of drone data, the model finally achieve to predict in a very high accuracy and exactness. in the meanwhile, for the tracker csrt was used.

  2. hardware

    for the hardware on the other hand, there are a lot of requirements, some of which are processing the frames, streaming the processed images to the ground station, getting the data of the drone each second and also live-streaming it to the ground station as well, keeping the connection of the device and the flight controller at the same time. in order to handle all the mentioned requirements, jetson xavier nx was decided to be embedded to the system. nvidia® jetson is the world’s leading platform for ai at the edge. its high-performance, low-power computing for deep learning and computer vision makes it the ideal platform for compute-intensive projects.

    while having dozens of options for the flight controllers, pixhawk cube 2 black was the personal choice due the experience and the reliability that it gives to the user, us in this case. the cube autopilot is a further evolution of the pixhawk autopilot. it is designed for commercial systems and manufacturers who wish to fully integrate an autopilot into their system.

    for the rest, basically the motors, esc’s and the others over average quality products were in order to minimize the number of the further potential issues regarding hardware.

problems

while all the tests being successfully accomplished both on virtual simulations and also f2f before the actual competition started, all the checks were also done accordingly and correctly. however, most of the problems are faced during the competition itself as almost most of them were unexpected.

hardware:

  1. low quality camera
  2. video transmission
  3. telemetry | receiver
  4. 433mhz specific
  5. hardware specific | esc or motors

software:

  1. cpp gui image processing and streaming | ffmpeg

solutions to the problems

  1. while at first having really hq fpv camera, which was caddx tarsier, due to the first failure in the drone and ground station connection, unfortunately, the biggest damage a drone’s fall over 10m’s was the loss of the camera completely. while without the camera you can not attend the competition at all, a new camera was in need to be acquired and embedded to the system asap. multiple previously purchased and owned action cameras were attempted to be used, however, due to the incompatibilities in the system, none of them were really supporting the webcam mode, which left us only 1 option: “the cheapest and for no reason brought years of old webcam”.

  2. starting with, a module named as ts832 & rc832 set was used in order to transmit the frames captured over the drone to the ground station, but the quality and the stability was at the lowest in the area due to the frequencies being crashed from the multiple opponents’ sources. to resolve this problem, rocket m5 alongside its power beam were purchased. while integration and setup processes were extremely easy, it was also very underrated transmission system imo. while almost having no latency, with the power of the beam, extending the distance of the transmission was as easy as pie.

  3. as it was also mentioned by the staffs of the competition that 915mhz transmitter and receiver were banned in usage in the challenge, it left the option of using 433mhz only. but, considering that all the other dozens of competitors are also using the same 433mhz transmitter, you were supposed to get used to the disconnections several times on top. tv channels streaming right next you, multiple dji drones flying also around you to record content to publish really did crush all of the plans. it was exceedingly hard to connect to our own device and control it. luckily, usage of 915mhz transmitter were condoned by the staffs later on, so everybody including us were forced to switch to it, which is still completely illegal to use.

  4. lastly, after fixing all of the problems described above, while thinking the device was completely ready to rumble, regrettably, it was not. in the first try to fly the drone fully autonomously in the center of the competition area, 2 of the motors on the same line/side did not move at all while the rest 2 working at the highest speed to take off the drone, with a very high force overturned the drone to the ground and 2 of the propeller were damaged so hard, they were in need to be replaced, however, as the spare propellers were stuck at the customs, we had to resign from the competition. still the reason of such a catastrophe is undiscovered, and as a team it is still being analyzed through the saved logs.

finally

that is pretty much our journey in teknofest 2021, which was full of unfortunates. while gaining a ton of experience though, i strongly believe placing first is going to be feasible in the next year.