Capturing Webhooks in Drupal 8

When using traditional APIs your application is typically requesting or pulling data from an external service, requiring a request for fresh data if you want to see recent changes. When using webhooks, that process is reversed: data is pushed from an external service in real-time keeping your application more up to date and your project running more efficiently. Here are a few examples:

• Facebook - Receive an alert anytime a message is read
• Stripe - Get alerted anytime a transaction comes through
• Eventbrite - Get alerted if an event is created or updated

This of course is not an exhaustive list; you'll need to check the application you are integrating with to see if they are implementing webhooks. A Google search like "Stripe Webhooks" is a good first step.

Implementing a webhook in your application requires defining a URL to which your webhook can push data. Once defined, the URL is added to the application providing the webhook. In Drupal 8, controllers are a straightforward way to define a path. See the complete code for an example.

When the webhook is fired it hits the defined URL with applicable data. The data that comes from a webhook is called the payload. The payload is often a JSON object, but be sure to check the application’s documentation to see exactly what you should be expecting. Capturing the payload is straightforward using the Request object available in a controller like this:

public function capture(Request $request) {
  $payload = $request->getContent();
}

If your payload is empty, you can always try some vanilla PHP:

$payload = file_get_contents("php://input");

Inspecting the Payload

Debugging webhooks can be a bit challenging if you are developing locally because your local environment typically does not have a public URL. Further, some webhooks require that the receiving URL implement SSL, which can also present challenges locally. The following options can help you navigate debugging webhooks locally.

Easiest

When capturing the payload, you can log it in Drupal. This option requires pushing your code up to a publicly available URL (a dev or staging environment).

$this->logger->debug('<pre>@payload</pre>', ['@payload' => $payload]);

Once you know what the payload looks like, you can copy it, modify it and make your own fake webhook calls locally using Postman. Feel free to checkout the importable Postman example in the repo.

Most Flexible

There is a utility called ngrok that allows you to expose your local environment with a publicly available URL; if you anticipate a lot of debugging it is probably worth the time to set up. Once ngrok is in place, you use the same logging method as above or use XDEBUG or something similar to inspect the payload. Ngrok will give you a unique, public URL which you can register, but which forwards to a server you have running on localhost. You can even use it with a local server that uses vhosts, such as yoursite.test with the command:

ngrok http -host-header=rewrite yoursite.test:80

Capturing and Processing the Payload

I'm a big fan of Drupal's queue system. It allows quick storage of just about anything (including JSON objects) and a defined way to process it later on a CRON run.

In your controller, when the payload comes in, immediately add the payload to your defined queue rather than processing it right away. This will make sure it is always running as efficiently as possible. You can of course process it right away if you choose to do so and skip the rest of this post.

$this->queue->createItem($payload);

Later when the queue runs, you can process the payload and do what you need to do, like create a node. Here is an example from the queue plugin (see ProcessPayloadQueueWorker.php for the full code):

 public function processItem($data) {   
   // Decode the JSON that was captured.
   $decode = Json::decode($data);
   // Pull out applicable values.
   // You may want to do more validation!
   $nodeValues = [
     'type' => 'machine_name_here',
     'status' => 1,
     'title' => $decode['title'],
     'field_custom_field' => $decode['something'],
   ];
 
   // Create a node.
   $storage = $this->entityTypeManager->getStorage('node');
   $node = $storage->create($nodeValues);
   $node->save();
 }

Once a queue is processed on CRON, the item is removed from the queue. Check out Queue UI module for easy debugging.

Security

As when building any web application, security should be a major consideration. While not an exhaustive list, here are a few things you can do to help make sure your webhook stays secure.

• Check your service's webhook documentation to see what authentication protocols they provide.
• Create your own token that only your application and the webhook service know about. If that is not included, do not accept the request. See the authorize method in the controller.
• Instead of processing the payload and turning it into a node, consider doing an API call back to the service using the ID from payload and requesting the data to ensure its authenticity.
• You should consider sanitizing content coming from the payload.

Once you implement a Webhook, you'll be hooked! Here's all the code packaged up.

There are of course Drupal contrib modules around webhooks. I encourage you to check them out, but if you have specific use cases or complex needs, rolling your own is probably the way to go.