RealSense - Builds a 3D World?

[jcowgar]

Quoted from the Yuneec Website:

Additionally, RealSense is capable of remembering its environment, further enhancing the prevention of possible collisions. It is not reactionary – if it avoids an obstacle once, it will remember the location of the obstacle and automatically know to avoid it.
​

Intel® RealSense™ Technology - Yuneec

I decided to give this a try. I took off in Angle mode, turned on Obs Avoidance, then proceeded to fly into the direction of my house (very, very slowly). Of course, Obs Avoidance started lighting up and stopped the H from going any further forward. I then moved left and right "mapping" out the house.

Now came the real test, would it remember the location of the obstacle and avoid it? I turned the H around so the back of the H was facing the house. I could backup to my hearts desire and nothing. Obs Avoidance was still on.

I know that the RealSense can not detect the object going backward, but I very clearly, and deliberately "introduced" my house to my H, didn't even move anywhere, I simply turned around and tried to go backward. It would have happily let me crash into it.

What exactly does it mean that it remembers its environment, is not reactionary, remembering the location and knows how to avoid it?

 

[Rayray]

Not in Angle Mode. Nothing is built in Angle Mode.

 

[CC Rider]

Quoted from the Yuneec Website:

Additionally, RealSense is capable of remembering its environment, further enhancing the prevention of possible collisions. It is not reactionary – if it avoids an obstacle once, it will remember the location of the obstacle and automatically know to avoid it.
​

Intel® RealSense™ Technology - Yuneec

I decided to give this a try. I took off in Angle mode, turned on Obs Avoidance, then proceeded to fly into the die of my house (very, very slowly). Of course, Obs Avoidance started lighting up and stopped the H from going any further forward. I then moved left and right "mapping" out the house.

Now came the real test, would it remember the location of the obstacle and avoid it? I turned the H around so the back of the H was facing the house. I could backup to my hearts desire and nothing. Obs Avoidance was still on.

I know that the RealSense can not detect the object going backward, but I very clearly, and deliberately "introduced" my house to my H, didn't even move anywhere, I simply turned around and tried to go backward. It would have happily let me crash into it.

What exactly does it mean that it remembers its environment, is not reactionary, remembering the location and knows how to avoid it?

Maybe I'm missing something.

You flew your Typhoon FORWARD, sensors toward your house, to "introduce" it to it; walked along your house "mapping" it out, then backed the Typhoon - sensors now facing AWAY from the house, Real Sense, too - and think that tested out the system? AM I missing something?

 

[jcowgar]

Maybe I'm missing something.

You flew your Typhoon FORWARD, sensors toward your house, to "introduce" it to it; walked along your house "mapping" it out, then backed the Typhoon - sensors now facing AWAY from the house, Real Sense, too - and think that tested out the system? AM I missing something?

The website says that the Typhoon maps out your environment and remembers obstacles. So, yes, I was attempting to test what that actually meant and how well it mapped out and remembered the 3D environment it was flying in.

What does the website mean when it says "if it avoids an obstacle once, it will remember the location of the obstacle and automatically know to avoid it."

That's what I was trying to learn and am interested in any information here. It seems that if it only remembers things when the RealSense can see it, then it didn't really remember it?

 

[CC Rider]

To my understanding of it, it means that the Typhoon with RS will map out the territory it is going to fly through first, avoid objects as it goes along its way (which is why it flies much slower), and IF it returns through that area again WHILE STILL POWERED ON it will intelligently avoid those same obstacles. It has no hardware to retain knowledge of where those obstacles exist in time and space, though. So it's not like, for example, Siri or any other voice recognition software that heaps levels upon levels of experiences on itself and truly "learns." It's once and done, over and out. Is that what you mean?

 

[jcowgar]

Not in Angle Mode. Nothing is built in Angle Mode.

In Angle Mode it will stop forward movement if Obs. Avoidance is on. It will not automatically navigate around it though, but it will stop you from hitting an object.

 

[jcowgar]

To my understanding of it, it means that the Typhoon with RS will map out the territory it is going to fly through first, avoid objects as it goes along its way (which is why it flies much slower), and IF it returns through that area again WHILE STILL POWERED ON it will intelligently avoid those same obstacles. It has no hardware to retain knowledge of where those obstacles exist in time and space, though. So it's not like, for example, Siri or any other voice recognition software that heaps levels upon levels of experiences on itself and truly "learns." It's once and done, over and out. Is that what you mean?

Yes, sorta. I guess the website is misleading a bit because it specially says it will remember the 3D world it is flying through. I did not shut down, nor land, nor even move really. The whole test was only about 1 minute long. I was just trying to learn through experimentation what the website actually is trying to say and hopefully start a conversation with other RealSense pilots as to their experiences. There isn't a lot of written material on the matter right now.

 

[CC Rider]

The Typhoon must be in Smart Mode for the 3D modeling you are expecting to be performed. That's one of the features of being in Smart Mode, and again why the Typhoon moves considerably slower when capable of using the features you are expecting. Angle Mode does not offer the many features that Smart Mode does; it's for a different kind of flying. And RS does not give you a 360-degree perimeter of OA around you. It only works within the viewing range of its sensors and is consistent only while still being powered on after it HAS rendered a 3D view of its surroundings.

Does that make sense to you? Am I explaining what you're looking to learn?

 

[jcowgar]

The Typhoon must be in Smart Mode for the 3D modeling you are expecting to be performed. That's one of the features of being in Smart Mode, and again why the Typhoon moves considerably slower when capable of using the features you are expecting. Angle Mode does not offer the many features that Smart Mode does; it's for a different kind of flying. And RS does not give you a 360-degree perimeter of OA around you. It only works within the viewing range of its sensors and is consistent only while still being powered on after it HAS rendered a 3D view of its surroundings.

I was not expecting the H to fly around the house while in Angle mode. I was hoping, but not expecting, it to know the house was there. OA works in Angle Mode just fine. It will stop your H from flying into an object it can see. It will not intelligently route around the object while in Angle mode, that requires Smart Mode.

I am fully aware that RS can only see forward. What I was unsure of is how much RS remembers, as the website talks about. Also, I was not expecting it to remember the 3D world after it was powered down and figured it had a limited memory for such a thing.

Does that make sense to you? Am I explaining what you're looking to learn?

So what you are saying is that the Typhoon has the ability to remember objects it just scanned and is currently facing. It does not build a "3D world" for later reference, even moments later, as the website suggests, is that correct?

 

[Rayray]

. It does not build a "3D world" for later reference, even moments later, as the website suggests, is that correct?

It does not in Angle Mode, nor does Yuneec say it does.

 

[CC Rider]

Once it has scanned an area and made a 3D model of it, and then if it is re-introduced into that same environment again, yes it is designed to re-route itself again as necessary to avoid those same KNOWN objects and obstacles. If something new is introduced into the environment then its 3D model may no longer hold true. If the unit is powered off, there goes its memory of the 3D model it created. So it is not intuitive. It temporarily "learns" its environment and will navigate around and through that environment as per the 3D model it has created. But NOT in Angle Mode.

 

[jcowgar]

Once it has scanned an area and made a 3D model of it, and then if it is re-introduced into that same environment again, yes it is designed to re-route itself again as necessary to avoid those same KNOWN objects and obstacles. If something new is introduced into the environment then its 3D model may no longer hold true. If the unit is powered off, there goes its memory of the 3D model it created. So it is not intuitive. It temporarily "learns" its environment and will navigate around and through that environment as per the 3D model it has created. But NOT in Angle Mode.

Thank you for the explanation. I think on my next test I will have it follow me in smart mode, see how long it takes to go around a simple object, then go right through that same area again and see if it does a better job the next time. Maybe that is a better test of its 3D mapping/learning capabilities. I started off too complex.

Also, I am well aware that Angle mode does not offer routing. It does offer obstacle avoidance though.

 

[CC Rider]

No problem, buddy. You are very welcome. Hope I helped you out some.

Yes, that is right about Angle Mode. And your next planned test would be a better, more qualified test of the Typhoon's RS capabilities. If you could, please consider posting your test results here when you're done. I'd be curious to see what you come up with. Good luck and take it easy. Enjoy your night.

 

[D See Ker]

if the 3D world was actually being built it should work in Angle mode, since GPS coordinates in this virtual environment would be enough to make it stop.

But I'm afraid the description is truly misleading, in Smart Mode, what happens is that the H always points the sensor in the forward flight direction, you will see the H rotating towards the direction you're turning the sticks towards. so even if it's creating a 3D depth map, it doesn't really build a map and navigate using it

 

[THoff]

Just because it turns the sensors in the direction of movement doesn't mean it doesn't have a model of what it saw previously, and utilizes that for decision-making.

If it didn't turn the sensors in the direction of flight, it wouldn't be able to deal with dynamic environments and avoid things that weren't there previously.

 

[Jester]

I was not expecting the H to fly around the house while in Angle mode. I was hoping, but not expecting, it to know the house was there. OA works in Angle Mode just fine. It will stop your H from flying into an object it can see. It will not intelligently route around the object while in Angle mode, that requires Smart Mode.

I am fully aware that RS can only see forward. What I was unsure of is how much RS remembers, as the website talks about. Also, I was not expecting it to remember the 3D world after it was powered down and figured it had a limited memory for such a thing.



So what you are saying is that the Typhoon has the ability to remember objects it just scanned and is currently facing. It does not build a "3D world" for later reference, even moments later, as the website suggests, is that correct?

I get what you're saying...If it is supposed to build itself a 3 dimensional map on it's way, well enough to back up, move over, or up, or down, and then get back on track because it built it's 3d environment enough to avoid them on it's rear and sides to do it the first time, but then didn't stop itself the second time, then it doesn't seem to truly remember it's 3D world for very long. We all know that RS is only on the front, but if it says it remembers it, and it was able to negotiate it's way back out and around the first time it found itself needing to, you would think it would now be able to go backward in the same spot it already observed, and know not to hit it, even if it's heading orientation has changed. Apparently it got it's memory from the same place I got mine. Bummer.

 

[Rayray]

Good grief...

In Follow Me and/or Watch Me Smart Modes, and only those modes, operating "hands off", RealSense indeed builds its 3D map.

Example: Following you on a bicycle down a path, then you turn around and go back. On the way back it navigates much better. The memory is only retained for the duration of that flight.

Whether anyone chooses to believe this or not is their business. I personally believe the Sun rises and sets, so there you go.

 

[RiverRunner]

I don't have Realsense on mine yet. I am curious about whether your test would work in smart mode. If you map the house pointed at it in smart mode then yaw 180 degrees and fly towards it, does it remember and compensate? Does it instead rotate the H to point in the direction it is traveling in smart mode with Realsense?

 

[Jester]

Good grief...

In Follow Me and/or Watch Me Modes, and only those modes, operating "hands off", RealSense indeed builds its 3D map.

Example: Following you on a bicycle down a path, then you turn around and go back. On the way back it navigates much better. The memory is only retained for the duration of that flight.

Whether anyone chooses to believe this or not is their business. I personally believe the Sun rises and sets, so there you go.

Sorry about that, RayRay. I completely overlooked that it was only in those modes that it builds it's map. I have read everything I can think of about the H, but it's a lot to take in, and since I haven't even gotten to that comfort level with the H and am still just working on perfecting my skills in angle mode, I haven't put RS into play enough to remember all of it's long list of impressive details and abilities. Didn't mean to annoy you, but that makes much more sense now, and only confirms to me that taking it all slow and careful like I have until I am at the skill level you guys are, is a real good approach for me. I suffer from a pretty sever case of ADD, (not joking) so remembering all the details of everything this crazy beast is capable of can seriously be too much for me, and I apologize for the irritation I most likely cause. Forgetting that those 2 modes are the only time it employs it's safety skills is my bad.

 

[Rayray]

Didn't mean to annoy you, but that makes much more sense now, and only confirms to me that taking it all slow and careful like I have until I am at the skill level you guys are, is a real good approach for me. I suffer from a pretty sever case of ADD, (not joking) so remembering all the details of everything this crazy beast is capable of can seriously be too much for me, and I apologize for the irritation I most likely cause. Forgetting that those 2 modes are the only time it employs it's safety skills is my bad.

Yikes, MY bad, I didn't intend to sound PO'd, just trying to be more clear with my info. As for the "3D map", it's a bit of a "mountain in a molehill", since it is what it is, and nothing we can do about it. I gotta enough real flying stuff to learn w/o worrying about theory.

I quite enjoy your sense of humor.