How To: Tello Drone To RoboRealm - Jack Pender, Blog #8

 


Introduction:

This blog post is a tutorial intended to give you everything you need in order to start streaming the camera feed from a Tello drone and get this stream into RoboRealm. All of the necessary files are included in this repository: https://github.com/jackpender/TelloToRoboRealm
I have detailed the steps but I suggest also taking a look at the repository as you go, especially the python file if you aren't familiar with OpenCV.

Python File:

·        The first step is to import all the libraries that we will use.



·        We will be livestreaming the tello camera feed to the localhost using flask, so we now initialize this flask app.



·        We also open up the connection to the drone; me = Tello(), setting its name to “me”.



·        We then create a function called “gen_frames” to continuously grab the image that the Tello camera is taking. These are essentially stitched together in order to create a video.

·        In “gen_frames”  we create a While loop, in which we get the frame from the Tello and then flip it around so it isn’t mirrored.



·        At the end of this While loop this flow of images is stitched together, resulting in the video that will be displayed in the localhost.



·        Between the input of frames from Tello and their stitching together, there is code in order to detect the colour green. This isn’t absolutely necessary but could potentially be implemented to control the flight of the Tello drone.

·        Lastly we deploy the flask application.



·        In order to get started with Tello and OpenCV I found this video very useful: https://www.youtube.com/watch?v=vDOkUHNdmKs

 

Roborealm:

·        Create a variable called "bootup" using Set_Variable like so.


·        Create a Display_Variables and have it display “bootup” in the top middle of the screen



·        Create a Button_Interface, with three buttons; reset, takeoff and land



·        This is done by pressing config, where you can assign each button a variable



·        To “reset” assign bootup as the variable, with a value of 0

·        You can now move the original Set_Variable to be after the Button_Interface

·        Now we can set up an If_Statement

·        Here, we set the variable as bootup, the Comparison as “= equals”, Comparison Type as Integer and finally the Value as 0. We can leave Join empty.



·        We can now set up a VBScript_Program in order to run our python code.



·        Select “Use The Following Text” and we can just write the Visual Basic code here.

            Dim objShell, strPath1, strAttr1 ', strAttr2, strAttr3

            Set objShell = CreateObject ("WScript.Shell")


            strPath1 = """C:\Python\Python39\python.exe"""

            strAttr1 = " droneColor.py "


            objShell.Run strPath1 & strAttr1

·        We set two string to be printed out to the terminal, leaving a space at the start of the second string so that it prints as a legitimate command

·        We then use Set_Variable again to set bootup to 1, so that we don’t have an infinite loop



·        Now you can split your screen and, with RoboRealm on the left and your browser on the right, opened on the “localhost/video_feed”

Yours won't livestream until all steps are complete so don't expect it to look like this


·        Using Screen_Capture we can record what is on our screen

·        Select “Coordinates” to be the corners of the tello video feed. Yours may be different to mine. You can also set the “Frequency” to ASAP so that the video feed comes through in real time.



·        Finally, using Socket_Client we can connect to the IP Address and Port that the Tello operates on and send it commands.



·        Set “Server Address” to 192.168.10.1 and set “Server Port” to 8889. In the Console portion, select the “Ascii” option. In the box beneath “Ascii”, enter “command” and press Send Now. This is important as this establishes the UDP connection from RoboRealm to the Tello. Now you can send any of the Tello commands in order to control it e.g takeoff and land. This is a list of all the available commands: https://dl-cdn.ryzerobotics.com/downloads/Tello/Tello%20SDK%202.0%20User%20Guide.pdf

·        In order to get the livestream started, simply send “streamon()”.



Comments

Post a Comment

Popular posts from this blog

How to: Tello drone to micro bit - Victor Alikberov, Blog#1

Image
1. Introduction. The purpose of this blog is to explain everything that is needed in order to connect tello drone to micro bit. It will cover wiring,  javascript code behind custom created tello extension and how to use this extension to connect and drive tello via microbit. 2. Wiring. Since tello drone creates its own wi-fi hotspot, additional wi-fi module is required in order to establish wireless connection between micro bit and tello. ESP8266 will be used to connect micro bit to tello wi-fi hotspot and also to establsih UDP communications between micro bit and drone. The pinout of the ESP8266 chip is shown below. ESP8266 chip pinout. Receiving port (Rx) is connected to transmitting port of the micro bit (pin 2) and transmitting port (Tx) of the ESP8266 is connected to receiving port of the micro bit (pin 1).  3.3 volts power supply should be provided both to micro bit and ESP8266 chip. Chip enable port is also connected to 3.3v power supply.  ESP8266 plus microbi...
Read more

First USV Test - Jack Pender, Blog #5

Image
Introduction: On the 18 th of March we held our first field test at Killurin Bridge in Wexford. We met there at 10 am, with a plan of finishing up at 2 pm, giving us plenty of time to test each aspect of the project we have been working on. It was great to see the USV on the water for the first time, and it helped to visualise it in the setting that its use is intended for. Testing: The main tests for this first field test were to trial the driving functionality; Manual Drive, Microbit Drive and Cloud Drive. Using the controller, the drive of the USV was successful and it was great to finally see its speed and mobility on the water. With the Manual Drive, controlled by Patryck, going according to plan, this allowed us to test the Microbit Drive, which would be controlled by Adam. Using two Microbits to control the direction and thruster power, Adam completed a successful drive of the USV. This was great to see as it meant that the control of the USV could be friendly to everyone...
Read more

OpenCV To RoboRealm - Jack Pender, Blog #7

Image
Introduction: Developing further on getting the camera feed from the Tello drone to a viewable format, it was decided to try and get the camera feed into RoboRealm. Once the camera feed is in RoboRealm, it can essentially be sent to anywhere from there, in our case Ubidots, but if this were to change we would now have a better starting point. RoboRealm: RoboRealm was created in 2006 to take advantage of lower cost generic computing, a widening range of lower cost imaging devices, an increasing need and usage of vision as a primary sensor device and the desire to come to a solution with an interactive user interface with minimal programming. This made it ideal for this project as it meant that for future iterations it would be quick and simple for someone else to get up and running with. Procedure: Using multiple of the built-in features of RoboRealm I created a programe that would start a screen capture in order to have RoboRealm capture the drone feed. Firstly, within an if ...
Read more