Full example of magritte REST config?

user_6292923 0
Viewed 301

We're setting up a rest integration via magritte-rest-v2.

The documentation covers auth and making rest calls, but there is nowhere will a full example config and I can't seem to get it to work.

Anyone have a working config they can share for reference? My usecase is very simple: token auth, and paginated list of returned objects.

palantir-foundry
4 Answers

Here is an example of a source with oauth2 call to retrieve an access_token, which is than placed into the header:

type: magritte-rest-v2
sourceMap:
  auth_source:
    type: magritte-rest
    url: 'https://auth.api.com/'
    timeoutInMinutes: 3
    proxy: 'http://proxy.com'
  data_source:
    type: magritte-rest-auth-call-source
    proxy: 'http://main.api.com/'
    url: 'http://proxy.com'
    headers:
      Authorization: 'Bearer {%token%}'
      Accept: application/json
    authCalls:
      - source: auth_source
        type: magritte-rest-call
        method: POST
        path: connect/token
        headers:
          Authorization: 'Basic {{basic_auth_secret}}'
        formBody:
          username: '{{username}}'
          password: '{{password}}'
          grant_type: password
          scope: customer-api
        extractor:
          - type: magritte-rest-json-extractor
            assign:
              token: /access_token
user_36528050

I don't have an example with pagination at hand, but here's one without. I'll edit this answer once I find one.

type: magritte-rest-v2
sourceMap:
  my_api:
    type: magritte-rest-auth-parameters-source
    url: 'https://api.yourserver.io/foo/bar/'
    parameters:
      api_key: '{{api_key}}'
user_260040

You'll want to distinguish between the authentication and the paging call. The authentication is defined by your source, which you can setup as fsmf shows:

type: magritte-rest-v2
sourceMap:
  my_api:
    type: magritte-rest
    headers:
      Authorization: 'Bearer {{token}}'
    url: "https://some-api.com/"

This will ensure all calls made to the API using this source have the token attached as a header.

Next you need to setup your sync, which will be responsible for making the relevant endpoint calls. The setup here depends on how your endpoint deals with pagination. In the documentation you'll find the sync examples

  • Page-based API
  • Offset-based API
  • Next-Page link-based API

Which cover the most common paging setups.

user_167339650

This config pulls back 50 objects per-page:

type: rest-source-adapter2
restCalls:
  - type: magritte-paging-inc-param-call
    method: GET
    path: search
    paramToIncrease: startAt
    increaseBy: 50
    initValue: 0
    parameters:
      startAt: '{%startAt%}'
    extractor:
      - type: json
        assign:
          issues: /issues
        allowNull: true
    condition:
      type: magritte-rest-non-empty-condition
      var: issues
    maxIterationsAllowed: 4096
cacheToDisk: false
oneFilePerResponse: false

It results in a raw dataset that looks like: raw rest dataset with paged results arrays

... where response is the full response JSON and the paged data is in response.issues.

user_4954150
Related