TOBIAS SCHOTTSTÄDT
FULL-STACK DEVELOPER
FROM KASSEL

In today's digital landscape, the security of websites and web applications is more important than ever. Cyber threats such as phishing, malware and other malicious activities are on the rise and can cause significant damage to organisations. It is therefore crucial to have tools that recognise and prevent such threats at an early stage. One of these tools is Urlscan.io, an online service that enables in-depth analysis of URLs. In this article, we take a detailed look at Urlscan.io and introduce my new Laravel package that simplifies the integration of this service into your applications.

Table-of-contents

• What is Urlscan.io?
• Main functions of Urlscan.io
• How Urlscan.io works
• Benefits for companies
• Introduction of the Laravel Urlscan package
• Installing the Laravel Urlscan package
• Configuration and usage
• Conclusion

What is Urlscan.io?

Urlscan.io is an online service that allows users to comprehensively analyse URLs. It acts as a public frontend for analysing websites, similar to Virustotal for files and hashes. When you submit a URL to Urlscan.io, the service visits the website, reads all resources and creates a detailed report on the behaviour of the page. This includes information about the loaded scripts, embedded content, network connections and much more. The main purpose of Urlscan.io is to identify potentially malicious websites and provide detailed information to help with security analysis.

Main features of Urlscan.io

Urlscan.io offers a range of powerful features that make it an indispensable tool in the security industry:

• URL analysis: Detailed examination of the entered URL, including all loaded resources and network connections.
• Screenshot creation: Automatic creation of screenshots of the scanned page for visual inspection.
• Domain information: Display DNS information and SSL certificate details to identify forged or compromised domains.
• API access: Programmatic access to scans and results via a RESTful API for automated processes.
• Integration: Ability to integrate with SIEM systems and other security platforms for a seamless security infrastructure.

How Urlscan.io works

Urlscan.io uses a virtual browser instance to visit the entered URL. During the visit, the service logs every network interaction and resource that the website loads. This process enables in-depth analysis, including the detection of hidden JavaScript, unexpected redirects and suspicious domain connections. The collected data is then summarised in a detailed report that provides valuable insights for security researchers and companies.

Urlscan.io offers considerable advantages for companies:

• Threat detection: Early identification of phishing sites and malware to prevent security breaches
• Compliance: Support for compliance with security policies and standards through regular monitoring.
• Automation: Integration into existing workflows to automatically monitor and respond to threats.
Security research: Provision of data for in-depth analyses and development of security strategies.

Introduction of the Laravel Urlscan package

In order to facilitate the integration of Urlscan.io into Laravel applications, I have developed the new package laravel-urlscan. This package enables developers to integrate the functions of Urlscan.io directly into their projects and use the API seamlessly. It provides a simple and intuitive interface for performing URL scans, retrieving results and integrating with existing security processes.

The package is installed via Composer and is completed in just a few steps:

1. Package installation: Execute the following command:

   composer require xchimx/laravel-urlscan

2. Package installation: Add the service provider and the facade to your app.php configuration file if you are not using package detection:

   
   // config/app.php
   
   'providers' => [
       ...
       Xchimx\LaravelUrlScan\UrlScanServiceProvider::class,
       ...
   ];
   
   'aliases' => [
       ...
       'UrlScan' => Xchimx\LaravelUrlScan\UrlScan::class
       ...
   ];
   

3. Publish configuration file: Publish the configuration file with :

   php artisan vendor:publish --provider="Xchimx\\Urlscan\\UrlscanServiceProvider" --tag="config"

4. Set up API key: Add your Urlscan.io API key in the .env file:

   URLSCAN_API="YOUR-API-KEY-SET-HERE"

Configuration and usage

After installation, you can use the package as follows:

Basic use

Import the facade:

use Xchimx\LaravelUrlScan\UrlScan;

User

$user = UrlScan::user()->getQuotas();

Scan

$url = 'https://laravel.com/';
$visibility = 'public'; // Options: 'public', 'private', 'unlisted'
$result = UrlScan::scan()->submitUrl($url, $visibility);

Result

$uuid = '358c5c79-b712-4e61-b79e-4a59e3c8b116'; //laravel.com
$getResult = UrlScan::result()->getResult($uuid);
$getScreenshot = UrlScan::result()->getScreenshot($uuid);

Search

$query = 'page.url.keyword:https\:\/\/www.paypal.com\/*';
$getSearchResults = UrlScan::search()->search($query);

Combined example

    public function startScan()
    {
        $url = 'https://laravel.com/';
        $visibility = 'public'; // Options: 'public', 'private', 'unlisted'
        $result = UrlScan::scan()->submitUrl($url, $visibility);

        if (isset($result['uuid'])) {
            sleep(10); //necessary else the scan isn't finished yet
            $getResult = UrlScan::result()->getResult($result['uuid']);
            $getScreenshot = UrlScan::result()->getScreenshot($result['uuid']);
            return [
                'result' => $getResult,
                'screenhots' => $getScreenshot
            ];

        } else {
            return response()->json(['error' => 'UUID not found'], 400);
        }
    }

This example sends a scan request to Urlscan.io and returns the results. Takes the generated UUID and retrieves detailed information, it also creates the links to the screenshots.

Conclusion

Urlscan.io is a powerful tool for analysing the security of websites. With the new Laravel package, integration into your applications is easier than ever. Organisations can benefit from this integration by strengthening their security measures and detecting potential threats early. I recommend integrating this package into your security strategy to ensure a proactive approach to cyber defence.

The Github repository.

The repository on Packagist.