When I was learning about Azure AI, I asked myself the question: “how far is AI? What can it determine – and what is impossible (yet)?”
A nice tool that can give (partly) answers to that, is the website that Azure offers to scan images, it uses Azure Cognitive Services (computer vision): ( https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/#features ). It’s a nice website: you simply insert the photo url and then the result is shown.
Police car
I tested it on some images of the news site https://nu.nl. I got the link from one of their web pages (F12, then search for jpg):
https://media.nu.nl/m/q9mxv4fasfgx_wd1280.jpg/dode-bij-steekpartij-in-flat-heemskerk.jpg
When you paste this link and then press “Submit”, you will see rectangles around the main objects on the photo:
On the right side, you see the conclusions that are drawn:
| Objects | [ { “rectangle”: { “x”: 671, “y”: 407, “w”: 124, “h”: 76 }, “object”: “Vehicle registration plate”, “confidence”: 0.555 }, { “rectangle”: { “x”: 224, “y”: 159, “w”: 671, “h”: 507 }, “object”: “car”, “parent”: { “object”: “Land vehicle”, “parent”: { “object”: “Vehicle”, “confidence”: 0.926 }, “confidence”: 0.924 }, “confidence”: 0.876 } ] |
| Tags | [ { “name”: “car”, “confidence”: 0.9668071 }, { “name”: “land vehicle”, “confidence”: 0.964869 }, { “name”: “vehicle”, “confidence”: 0.950910568 }, { “name”: “wheel”, “confidence”: 0.702893734 }, { “name”: “vehicle registration plate”, “confidence”: 0.685587 }, { “name”: “full-size car”, “confidence”: 0.682034552 }, { “name”: “stage”, “confidence”: 0.6383412 }, { “name”: “text”, “confidence”: 0.6330378 }, { “name”: “mid-size car”, “confidence”: 0.5292624 }, { “name”: “blue”, “confidence”: 0.4259651 }, { “name”: “light”, “confidence”: 0.175804853 } ] |
| Description | { “tags”: [ “stage”, “table”, “sitting”, “car”, “blue”, “small”, “light”, “parked”, “truck”, “room”, “airplane”, “laptop”, “computer”, “night”, “street”, “bus”, “standing”, “man”, “bed”, “plane” ], “captions”: [ { “text”: “a blue car on a stage”, “confidence”: 0.6204072 } ] } |
What we see on the photo, is obvious for everyone in the Netherlands: it’s a police car in action. Azure did understand the vehicle registration plate, the fact that it is a car, and that it is a land vehicle. The meaning of the stripes on the car and the meaning of the blue light were missed.
Donald Trump
It’s nice to play around a little bit with different photos. Let’s take this one:
It does recognize Donald Trump, it also recognizes the smile on his face. Trump is wearing a suit and a tie is visible (correct). It also sees a phone (where?!) and as humans we see that he is waving (the recognition software doesn’t recognize this).
Conclusion
My conclusion: sometimes it’s impressive, sometimes it misses the clue.
Have fun with your own photos!
