Session Catalog for Oracle OpenWorld & CodeOne 2018 in JSON files

The Oracle OpenWorld and CodeOne 2018 conferences took place in the last week of October 2018. Together, over 1500 sessions took place. The session catalog with details for all sessions is accessible at https://events.rainfocus.com/widget/oracle/oow18/catalogcodeone18 – at least at the time of writing. The session information includes title and abstract, track, session type, names of speakers, download link for the slides and more.

The data in the session catalog represents an interesting data set that could easily be used for data visualization, machine learning, business analytics and demos of web & mobile & chat application. Therefore, it could be convenient to have easy access to the data set in the session catalog.

By inspecting the HTTP calls made from the Session Catalog Web Application, it is easy to learn how the session data can be retrieved from the REST API that exposes it.

With a fairly simple Node application, HTTP requests can be made to retrieve all session data per session type and per batch of 50 sessions – see https://github.com/lucasjellema/Oracle-OpenWorld-CodeOne-2018-SessionCatalog/blob/master/produce-oow-catalog.js. Note how function delay(t) is used to schedule a function call.

Executing this Node program for all session types results in JSON files with session details for session type. The data in these JSON files looks like this in an online JSON viewer:

This data can be used for various applications and demonstrations.

A more useful file can be compiled – a file that contains all sessions and for all sessions only relevant details (and for example not all internal identifiers). This file is oow2018-sessions-catalog.json . It has been created by the Node program  compile-oow-catalog-files.js. This program leverages the JSONata language for retrieving data from complex JSON documents as well as transforming JSON documents into target documents with a different structure. JSONata is comparable in function to XPath and XSLT or XQuery. The npm module JSONata was used in this case and proved very useful.

var catalog = []
// here follows the JSONata expression - equivalent to the XSLT template or XQuery mapping
var expression = jsonata(
    ` $.{
        'code': code
        ,'title': title
        ,'abstract': abstract
        ,'duration': length
        ,'event': eventName
        ,'type': type
        ,'waitlistPeak' : waitlistLimit
        ,'speakers': participants.{'name':(firstName & ' ' & lastName)
                                  , 'company': companyName
                                  , 'jobTitle': jobTitle
                                  , 'biography' : bio
                                  , 'photoURL' : photoURL
                                  , "twitter": attributevalues[attribute_id='twitter'].value
                                  , "designations": attributevalues[attribute_id='specialdesignations'].value
                                  }
        , 'slots' : times.{ 'room': room
                          , 'date': date
                          , 'time': time   
                          , 'roomCapacity' :capacity                  
                          }                             
        , 'levels' :  attributevalues[attribute_id='SessionsbyExperienceLevel'].value
        , 'roles' :  attributevalues[attribute_id='SessionsbyYourRole'].value
        , 'track' :  attributevalues[attribute_id='Track'].value                  
        , 'slidesToDownload' : files. 
                               { "name": filename
                               ,'url':url
                               }
    }
    `);

for (sessionType of sessionTypes) {
    var file = oow2018Filename + sessionType + ".json"
    if (fs.existsSync(file)) {
        console.log("Processing " + sessionType)
        var buf = fs.readFileSync(oow2018Filename + sessionType + ".json");
        var sessions = JSON.parse(buf)

        var result = expression.evaluate(sessions);
        catalog = catalog.concat(result)
    }//if
}//for
fs.writeFileSync(oow2018Filename + '.json', JSON.stringify(catalog, null, '\t'));

Resources

GitHub repo with the Node source code as well as the JSON files created for the session catalog data: https://github.com/lucasjellema/Oracle-OpenWorld-CodeOne-2018-SessionCatalog

Online JSON Inspector: http://jsonviewer.stack.hu/

Earlier article on retrieving Session Catalog for Oracle OpenWorld and JavaOne 2017: https://technology.amis.nl/2017/08/10/when-screen-scraping-became-api-calling-gathering-oracle-openworld-2017-session-catalog-with-node/

JSONata – home page: http://jsonata.org/

JSONata – in browser try out page – http://try.jsonata.org/