Browser

This Aptitude, combined with the Olive Helps browser extension, allows an author to interact with your browser.

This Aptitude only works if you have the Olive Helps browser extension installed. Currently, we have extensions for Chrome, Firefox, and Edge.

Please note: Microsoft Edge operates as a Chrome plugin.

These documents will reference frameId and parentFrameId - for more information on frames, check out the Chrome Extension Docs here.

API

listenNavigation

Calls callback on any navigation event pushed from a browser running the Olive Helps extension.

import { browser } from '@oliveai/ldk';

const URL_TO_OPEN = 'https://www.oliveai.dev/';
const newTabId = await browser.openTab(URL_TO_OPEN);

const browserListener = await browser.listenNavigation((navDetails) => {
    /*
    navDetails will be:
    {
        frameId: number,
        navigationType: string (real | history)
        parentFrameId: number,
        tabId: number,
        timestamp: number,
        url: string
    }
    */
    
    // navDetails.url === URL_TO_OPEN: true
    // navDetails.tabId === newTabId: true

    // Cancel the listener when a specific URL is opened by the user
    if (navDetails.url === URL_TO_OPEN) {
        browserListener.cancel();
    }
})

listenTextSelection

Calls callback on any text selection event pushed from a browser running the Olive Helps extension.

import { browser } from '@oliveai/ldk';

const browserTextListener = await browser.listenTextSelection((value) => {
    // Cancel the listener when a specific string is selected in the browser
    const TEXT_WE_NEED = 'The quick brown fox jumped';
    if (value === TEXT_WE_NEED) {
        browserTextListener.cancel();
    }
}

listenNetworkActivity

Listens for any network activity by the browser. The resulting callback for the listener takes in an activity event with the following structure.

export interface NetworkActivityDetails {
  tabId: number; // Which tab the activity originated from
  requestUrl: string; // The url for fetching data
  method: string; // The HTTP method used in the request
  requestBody: null | string; // Null or a string of the request payload
  domain: string; // The originating url
}

import { browser } from '@oliveai/ldk';

const networkActivityListener= await browser.listenNetworkActivity((event) => {
    console.log(JSON.stringify(event)); // This will produce the structure above
    networkActivityListener.cancel();
}

openTab

Opens a tab in the browser running the Olive Helps extension.

import { browser } from '@oliveai/ldk';

const URL_TO_OPEN = 'https://www.oliveai.dev/';

// This will open a new tab in your browser, and return the new tab's ID
const newTabId = await browser.openTab(URL_TO_OPEN);

// This will open a new tab in your browser and return both the tab id and the source html for the page
const {id, sourceHTML}= await browser.openTab(URL_TO_OPEN, {includeSouce: true});

openWindow

Opens a window in the browser running the Olive Helps extension.

import { browser } from '@oliveai/ldk';

const URL_TO_OPEN = 'https://www.oliveai.dev/';

// This will open a new window in your browser, and return the new window's ID
const newWindowId = await browser.openWindow(URL_TO_OPEN);

// This will open a new window in your browser, and return the new windows id and the source html for the page
const {id, sourceHTML} = await browser.openWindow(URL_TO_OPEN, { includeSource: true});

sourceHTML

Retrieves the html source for a given address.

import { browser } from '@oliveai/ldk';
const URL_TO_READ = 'https://www.oliveai.dev/';
const { id, sourceHTML } = await browser.sourceHTML(URL_TO_READ);

Example

In this example, let's assume we have a Whisper with a button, this instructs you to click the button in order to open a browser window that automatically navigates to your profile page for your insurance provider. And after doing so, you'll instructed to select the insurance ID and group ID.

import { browser } from '@oliveai/ldk';

export async function getInsuranceInfo() {
    // Assume we know the 9-digit ID will be on the page as 'ID: 123456789'
    // and the 8-digit group as 'Group: 12345678'
    const ID_SELECTION_PATTERN = /ID: .{9}/g
    const GROUP_SELECTION_PATTERN = /Group: .{8}/g
    const CIGNA_USER_PROFILE_URL = 'https://my.cigna.com'

    await browser.openWindow(CIGNA_USER_PROFILE_URL);

    let insuranceId;
    let insuranceGroup;

    const browserListener = await browser.listenTextSelection(value => {
        const idMatches = value.match(ID_SELECTION_PATTERN);
        const groupMatches = value.match(GROUP_SELECTION_PATTERN);

        // If there is a match, assign the variable
        if (idMatches) insuranceId = idMatches[0];
        if (groupMatches) insuranceGroup = groupMatches[0];
        
        // If insuranceId and insuranceGroup are defined, we can close
        // the listener and continue with the rest of the logic of our loop
        if (insuranceId && insuranceGroup) {
            browserListener.cancel();
            // Create/update whispers or execute other logic here
            // using the ID and Group
        }
    })
}

Permissions

To use the Browser aptitude, you will need to specify which domains your Loop will be using in your package.json under the ldk object like shown below. If you do not specify a domain your Loop attempts to access, Olive Helps will reject the request. This is done to ensure you feel confident that a Loop isn't sending or getting information to something unexpected.

The URLs you add will need to include a path suffix /*

...
"ldk": {
  "permissions":
  {
    "browser": {
      "urlDomains": [
        {
          "value": "*.fda.gov", // Allows any subdomain of fda.gov, but
        },                      // not any path
        {
           "value": "*.fda.gov/*", // Allows any subdomain and path of fda.gov
        },
        ...
      ]
    },
    ...
  }
},
...