Playing with AI in Azure

Frederique Retsema

When I was learning about Azure AI, I asked myself the question: “how far is AI? What can it determine – and what is impossible (yet)?”

A nice tool that can give (partly) answers to that, is the website that Azure offers to scan images, it uses Azure Cognitive Services (computer vision): ( https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/#features ). It’s a nice website: you simply insert the photo url and then the result is shown.

Police car

I tested it on some images of the news site https://nu.nl. I got the link from one of their web pages (F12, then search for jpg):

https://media.nu.nl/m/q9mxv4fasfgx_wd1280.jpg/dode-bij-steekpartij-in-flat-heemskerk.jpg

When you paste this link and then press “Submit”, you will see rectangles around the main objects on the photo:

On the right side, you see the conclusions that are drawn:

Objects[ { “rectangle”: { “x”: 671, “y”: 407, “w”: 124, “h”: 76 }, “object”: “Vehicle registration plate”, “confidence”: 0.555 }, { “rectangle”: { “x”: 224, “y”: 159, “w”: 671, “h”: 507 }, “object”: “car”, “parent”: { “object”: “Land vehicle”, “parent”: { “object”: “Vehicle”, “confidence”: 0.926 }, “confidence”: 0.924 }, “confidence”: 0.876 } ]
Tags[ { “name”: “car”, “confidence”: 0.9668071 }, { “name”: “land vehicle”, “confidence”: 0.964869 }, { “name”: “vehicle”, “confidence”: 0.950910568 }, { “name”: “wheel”, “confidence”: 0.702893734 }, { “name”: “vehicle registration plate”, “confidence”: 0.685587 }, { “name”: “full-size car”, “confidence”: 0.682034552 }, { “name”: “stage”, “confidence”: 0.6383412 }, { “name”: “text”, “confidence”: 0.6330378 }, { “name”: “mid-size car”, “confidence”: 0.5292624 }, { “name”: “blue”, “confidence”: 0.4259651 }, { “name”: “light”, “confidence”: 0.175804853 } ]
Description{ “tags”: [ “stage”, “table”, “sitting”, “car”, “blue”, “small”, “light”, “parked”, “truck”, “room”, “airplane”, “laptop”, “computer”, “night”, “street”, “bus”, “standing”, “man”, “bed”, “plane” ], “captions”: [ { “text”: “a blue car on a stage”, “confidence”: 0.6204072 } ] }

What we see on the photo, is obvious for everyone in the Netherlands: it’s a police car in action. Azure did understand the vehicle registration plate, the fact that it is a car, and that it is a land vehicle. The meaning of the stripes on the car and the meaning of the blue light were missed.

Donald Trump

It’s nice to play around a little bit with different photos. Let’s take this one:

https://media.nu.nl/m/h5jxdbbaaafn_wd1280.jpg/trump-neemt-abrupt-afscheid-van-twee-impeachment-advocaten.jpg

It does recognize Donald Trump, it also recognizes the smile on his face. Trump is wearing a suit and a tie is visible (correct). It also sees a phone (where?!) and as humans we see that he is waving (the recognition software doesn’t recognize this).

Conclusion

My conclusion: sometimes it’s impressive, sometimes it misses the clue.

Have fun with your own photos!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Next Post

Building my own Video Recorder–Automating Scheduled Streaming Video Recording on Windows 10

My objective: create a mechanism for recording streaming video on my Windows laptop on a preprogrammed timestamp and channel. My motivation:  show that I can do this (using AutoIt, OBS, Windows Task Scheduler an Windows Sandbox) and create a way around the limitation of my current TV provider (Ziggo in […]
%d bloggers like this: