AEM Implementation

Google provides an easy to use REST API for label identification (detecting individual objects on an image), landmarks detection, sentiment analysis, logo detection and amazing optical character recognition (OCR).  These services can be used within any application with access to images.

Into the details …

Google requires registration with the Vision Services and establishing an authentication scheme for authentication and/or authorization.  The simplest authentication scheme is an “API Key”.   This scheme provides a JSON file which can be referenced in the code for authentication.

We have created an AEM service to invoke Google’s API.  This service can be referenced by any component, service, or workflow process.  We extended the OOTB workflow “Update DAM Asset” to include the additional process of calling this service for every asset added to the DAM.

Our process determines the type of annotation coming back from Google (label, landmark, sentiment, or logo) and creates a tag in the corresponding taxonomy (see screenshots above).  The process also includes an OCR Vision call, which when made extracts the text identified on the image and populates the metafield “dc:description” on the DAM asset.

AEM “Smart Tags”

Adobe has also announced a native auto asset tagging feature called “Smart Tags” in its AEM 6.2 beta version which was received very well at the Adobe Summit—The Digital Marketing Conference 2016.  Though it is not available for public yet, it would be very interesting to compare it with Google Cloud Vision API’s once available. The Google Vision / AEM integration will work in any version of AEM, not just 6.2.

Conclusion

We are very excited to see what machine learning and AI are contributing to asset repositories in terms of classification. As we keep testing, we are amazed at the great results and disappointed with few misses (It seems to always fail to recognize the Nike logo). As Chad pointed out, the results are not always accurate and for a beta release of Cloud API, we are very impressed. We would now like to see how they compare with Adobe “Smart Tagging” native feature.