+ - 0:00:00
Notes for current slide
Notes for next slide

SoftLayer API

How it works, and what to do when it doesn't

Christopher Gallo, Senior API Developer

1 / 27

Christopher Gallo

  • Responsible for helping customers use the API
  • Maintain sldn.softlayer.com
  • Maintain the SLCLI (softlayer-python library), and other open source libraries
  • 10+ years working at SoftLayer, in a variety of roles.
2 / 27

Anatomy of an API Call

https://api.softlayer.com

  • Public Internet Endpoint.

https://api.service.softlayer.com

  • Private Network Endpoint
  • Only difference is the SoftLayer_Resource_Metadata service.
3 / 27

Specify the endpoint type and version

api.softlayer.com/rest/v3.1

Types

  • REST
  • SOAP
  • XMLRPC
  • Mobile

Versions

  • v3
  • v3.1

4 / 27

API is broken down into "Services", which contain a collection of methods that perform some action.


api.softlayer.com/rest/v3.1/SoftLayer_Service


Data returned by the API is broken down into "Datatypes". Services usually have a matching datatype, but not all data types have a matching service.


5 / 27

Methods tell the API which action to actually run in a class/service.

api.softlayer.com/rest/v3.1/ SoftLayer_Service/method

$ curl -s -u $SL_USER:$SL_APIKEY \
'https://api.softlayer.com/rest/v3.1/SoftLayer_Account/getObject.json'
{"accountManagedResourcesFlag":false,"accountStatusId":1001,"address1":"4849 Alpha Rd","allowedPptpVpnQuantity":0,"brandId":2,"city":"Dallas", LOTS OF DATA}
6 / 27

Some methods require an InitParamter, listed in the documentation. For REST calls, this goes between the Service and Method. For SOAP/XML requests, this is set in the header.

api.softlayer.com/rest/v3.1/SoftLayer_Service/ initParameter/method

$  curl -s  -u $SL_USER:$SL_APIKEY \
'https://api.softlayer.com/rest/v3.1/SoftLayer_Virtual_Guest/100317048/getObject'
{"accountId":307608,"createDate":"2020-04-07T04:01:49-06:00","dedicatedAccountHostOnlyFlag":false,"domain":"chechu.com", OTHER DATA}
7 / 27

You can control the format of the returned data by adding .FORMAT to the end of the URI. This is optional, JSON is assumed for REST calls.

api.softlayer.com/SoftLayer_Service/ initParameter/method.json

Available formats

  • json
  • txt
  • xml
8 / 27

Object Masks are specified as URL parameters, and take the form of a nested array, starting at the data type returned by the method.

api.softlayer.com/SoftLayer_Service/ initParameter/method ?objectMask=mask[id,name,relationalProperty[id, name]]

$ curl -s -g  -u $SL_USER:$SL_APIKEY  'https://api.softlayer.com/rest/v3.1/
SoftLayer_Virtual_Guest/100317048/getObject?objectMask=mask[id,hostname]'
{"hostname":"activedirectory","id":100317048}

-g for curl lets you use un-escaped brackets

-s for curl silences transport output

9 / 27

What is an object mask?

Returns a SoftLayer_Virtual_Guest datatype

Think of a datatype as a table in a database.

  • Local Properties
    • Selecting a local property "deselects" all the other default properties that would normally be returned.
    • This is ideal to reduce the size of the data returned.
  • Relational Properties
    • Selecting these "joins" in other tables to the query
    • Have their own set of local and relational properties
  • Count Properties
    • Returns a count of how many of these an object has.
10 / 27
mask[
  id, # local
  hostname, # local
  datacenter[ # relational, SoftLayer_Location datatype
    id, # local to SoftLayer_Location
    name, # local to  SoftLayer_Location
    timezone # relational, SoftLayer_Locale_Timezone
  ]
]
$ curl -s  -g  -u $SL_USER:$SL_APIKEY  'https://api.softlayer.com/rest/v3.1/
SoftLayer_Virtual_Guest/100317048/getObject
  ?objectMask=mask[id,hostname,datacenter[id,name,timezone]]'
{
  "hostname":"activedirectory","id":100317048,
  "datacenter":{
    "id":1854895, "name":"dal13",
    "timezone":{
      "id":114, "longName":"(GMT-06:00) America\/Dallas - CST",
      "name":"America\/Chicago", "offset":"-0600",  "shortName":"CST"}
  }
}
11 / 27

Object Filters

While object masks let you select which columns from a query you want. Object Filters let you select which rows from the query you want. These are essentially the WHERE clause for any given query.

Gotchas

  • Filters generally start the top level at the Service's datatype, NOT the return datatype.
  • Errors and Bad filters can be silently ignored.
  • Can only filter the number of rows in the returned datatype. Can not limit rows from any of the relational properties.
  • OPTIONS are an array of usually a single property.
12 / 27

ObjectFilter Example

curl -u $SL_USER:$SL_APIKEY 'https://api.softlayer.com/rest/v3.1/
SoftLayer_Account/getVirtualGuests.json
  ?objectFilter={"virtualGuests":{"hostname":{"operation":"adns"}}}'
[{"domain":"vmware.chechu.com","fullyQualifiedDomainName":"adns.vmware.chechu.com","hostname":"adns","id":100250634, OTHERDATA}}]
13 / 27

Result Limits

Allows you to 'page' through results. Only useable if the return datatype is an array (marked by [])

Added as a URL parameter like resultLimit=<OFFSET>,<LIMIT>

SoftLayer-Total-Items header tells you how many total items to expect.

curl -u $SL_USER:$SL_APIKEY 'https://api.softlayer.com/rest/v3.1/
  SoftLayer_Account/getTickets.json?resultLimit=0%2C10'


NOTE SQL still needs to compute the query, so adding a limit will not solve errors that are caused by SQL taking too long to return.

14 / 27

All of the Above

$ curl -s  -v -g  -u $SL_USER:$SL_APIKEY 
'https://api.softlayer.com/rest/v3.1/SoftLayer_Account/getVirtualGuests.json
?objectFilter={"virtualGuests":{"hostname":{"operation":"adns"}}}
&objectMask=mask[id,hostname]
&resultLimit=0,10'
[{"hostname":"adns","id":100250634}]
< HTTP/1.1 200 OK
< Date: Tue, 21 Apr 2020 21:44:04 GMT
< Server: Apache
< X-Frame-Options: SAMEORIGIN
< SoftLayer-Total-Items: 1
< Content-Length: 36
< Vary: Accept-Encoding
< Connection: close
< Content-Type: application/json
< Strict-Transport-Security: max-age=31536000
15 / 27

Parameters

  • Any method that takes in parameters requires a POST request.

  • Parameters are NOT named in the request, but order does matter. Should match the order from the documentation.

  • SoftLayer_Dns_Domain::createNsRecord()

curl -u $SL_USER:$SL_APIKEY -X POST 
-d '{"parameters": ["ns1.sltesting.com.", "192.168.1.1", 3600]}'
'https://api.softlayer.com/rest/v3.1/SoftLayer_Dns_Domain/
3217092/createNsRecord.json'
16 / 27
  • Some parameters are Objects or arrays themselves, and are specified in a JSON format

SoftLayer_Dns_Domain_ResourceRecord::createObject()

curl -u $SL_USER:$SL_APIKEY -X POST 
-d '{"parameters": [{"data": "126.123.123.1",
  "host": "test", "type": "NS", "ttl": 3600, "domainId": 3217096}]}'
'https://api.softlayer.com/rest/v3.1/SoftLayer_Dns_Domain_ResourceRecord/
createObject'

Basic Format

{"parameters":[
  "parameter1",
  {"parameter2": "is an object parameter"},
  [{"parameter3": "is an array"}, {"of other objects"}]
]}
17 / 27

SLCLI

SLCLI Docs SLCLI Project

$ slcli call-api --help
Usage: slcli call-api [OPTIONS] SERVICE METHOD [PARAMETERS]...
  Call arbitrary API endpoints with the given SERVICE and METHOD.
Options:
  --id TEXT                       Init parameter
  --mask TEXT                     String-based object mask
  --limit INTEGER                 Result limit
  --offset INTEGER                Result offset
  -v, --verbose                   Sets the debug noise level, specify multiple
                                  times for more verbosity.
  --format [table|raw|json|jsonraw] Output format  [default: raw]
  --json-filter TEXT              A JSON string to be passed in as the object
     filter to the API call. Remember to use  double quotes ("") for variable
     names. Can NOT be used with --filter. Dont use whitespace outside of 
     strings, or the slcli might have trouble parsing it.
18 / 27

SLCLI Debugging

$  slcli -vvv  call-api SoftLayer_Account getObject
Calling: SoftLayer_Account::getObject(id=None, mask='', filter='{}', args=(), limit=None, offset=None))
Starting new HTTPS connection (1): api.softlayer.com:443
https://api.softlayer.com:443 "GET /rest/v3.1/SoftLayer_Account/getObject.json HTTP/1.1" 200 5053
Returned Data:
{'accountManagedResourcesFlag': False, 'accountStatusId': 1001, 'address1': '4849 Alpha Rd',  LOTS OF OUTPUT }
execution_time    0.319053s
api_calls         SoftLayer_Account::getObject (0.207001s)
id      None
mask
filter  {}
limit   None
offset  None
curl -u $SL_USER:$SL_APIKEY -X GET -H "Accept: */*" -H "Accept-Encoding: gzip, deflate, compress"  'https://api.softlayer.com/rest/v3.1/SoftLayer_Account/getObject.json'
19 / 27

SLCLI Debugging

$ slcli -vvv --format=json call-api [SERVICE] [METHOD] 
[--id <ID>]
[--json-filter <JSON>] 
[--mask <mask[]>]
[--limit <limit>]
[--offset <offset>]
[param1]
[param2]
[param3]
[paramX]

Code Debugging

20 / 27

SLCLI XML

  • Debug output depends on the endpoint you are using.
    • REST endpoint will output a curl call.
    • XMLRPC endpoint will output a basic python program with payload.
21 / 27

IBMCLOUD SL

IBMCLOUD CLI has a function call sl which mimics the slcli.

Developed as a seperate product, but still supported to some extent. Has a similar call-api function.

IBMCLOUD_TRACE=true to enable API call output, all in REST.

22 / 27

Debugging

  • What you need to know before opening a support ticket
    • Code snippet using a supported library is fine, Pure curl call is better.
    • api user is helpful. Never write down API key.
    • Did this work in the past, or trying to get it to work.
    • How reproduceable is it?
    • The actual error the SL API returned.
    • SoftLayer_Exception_<WHATEVER>
    • Internal Server Error
23 / 27

Common Errors

  • Exception Handling Timeout Handling

  • SoftLayer_Exception_ObjectNotFound or SoftLayer_Exception_NotFound

    • Object doesn't exist, or user doesn't have access. Check initParameters
  • SoftLayer_Exception_WebService_BadRequest
    • Generic error meaning your request wasn't parseable. Check the objectMaks/objectFilter/parameters for formatting errors.
  • SoftLayer_Exception_Public / Internal Server Error
    • Generally means you asked for too much data. Check the objectMask, reduce number of relational properties. Use a result Limit
    • Sometimes can indicate a deeper problem, check splunk for more details.
  • Timeout
    • Query was too complicated for the database to calculate in time, was forceibly closed. Check object mask, or possible object filters.
24 / 27

DATA OVERLOAD

$> time curl -s -v -u $SL_USER:$SL_APIKEY -g \
'https://api.softlayer.com/rest/v3.1/SoftLayer_Account/getInvoices.json?objectMask=mask[id]'
< HTTP/1.1 200 OK
< SoftLayer-Total-Items: 7922
< Content-Length: 123508 (0.123508 MB)
real    0m1.163s
$> time curl -v -u $SL_USER:$SL_APIKEY -g 'https://api.softlayer.com/rest/v3.1/
SoftLayer_Account/getInvoices.json?objectMask=mask[id,invoiceTopLevelItems]'
< HTTP/1.0 200 OK
{"error":"Internal Error","code":"SoftLayer_Exception_Public"}
real    0m16.731s
$> time curl -v -u $SL_USER:$SL_APIKEY -g 'https://api.softlayer.com/rest/v3.1/
SoftLayer_Account/getInvoices.json?objectMask=mask[id,account]'
< HTTP/1.0 500 Internal Server Error
< SoftLayer-Total-Items: 7922
[5 bytes data] # returns nothing
real    0m8.428s
25 / 27

Data overload is the most common problem. Here are 3 API calls to the same method.

First works just fine. Second fails, but has a 200 OK HTTP code Third Fails with a 500 HTTP code, but no message.

DATA OVERLOAD

$> time curl -v -u $SL_USER:$SL_APIKEY -g 'https://api.softlayer.com/rest/v3.1/
SoftLayer_Account/getInvoices.json?objectMask=mask[id,invoiceTopLevelItems]&resultLimit=0,2000'
  0     0    0     0    0     0      0      0 --:--:--  0:00:09 --:--:--     0{ [5 bytes data]
  0     0    0     0    0     0      0      0 --:--:--  0:00:09 --:--:--     0
< HTTP/1.1 200 OK
< Date: Thu, 30 Apr 2020 22:03:35 GMT
< SoftLayer-Total-Items: 7925
< Content-Length: 1706185 ( 1.706185 MB)
100 1666k  100 1666k    0     0   165k      0  0:00:10  0:00:10 --:--:--  436k
real    0m10.103s
$> time curl -v -u $SL_USER:$SL_APIKEY -g 'https://api.softlayer.com/rest/v3.1/
SoftLayer_Account/getInvoices.json?objectMask=mask[id,account]&resultLimit=0,2000'
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0{ [5 bytes data]
  0     0    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--     0
< HTTP/1.1 200 OK
< Date: Thu, 30 Apr 2020 21:57:02 GMT
< SoftLayer-Total-Items: 7925
< Content-Length: 133522001 ( 133.522001 MB)
100  127M  100  127M    0     0  8221k      0  0:00:15  0:00:15 --:--:-- 10.7M
real    0m19.761s
26 / 27

First call takes 9 seconds to return, indicating a long time pulling data from the DB.

Second call takes 3 seconds to return, but a very large Content-Length, indicting too much data was returned.

Contributing

27 / 27

Christopher Gallo

  • Responsible for helping customers use the API
  • Maintain sldn.softlayer.com
  • Maintain the SLCLI (softlayer-python library), and other open source libraries
  • 10+ years working at SoftLayer, in a variety of roles.
2 / 27
Paused

Help

Keyboard shortcuts

↑, ←, Pg Up, k Go to previous slide
↓, →, Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
Number + Return Go to specific slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
Esc Back to slideshow