How can a scannable BLE beacon be implemented using the bluez stack?

I'm trying to implement a BLE beacon which allows for additional information to be requested.

My understanding so far is that in BLE, a device can broadcast advertisement packets. An advertisement packet can indicate that the device is scannable. This means that a client can send a scan request to the beacon. A scan response can then be sent by the beacon, containing additional information.

So the exchanged packets would look like this: ADV_SCAN_IND -> SCAN_REQ -> SCAN_RSP.

I'm trying to understand how the beacon implementation should behave. Is this something implemented by the adapter (I would have to specify upfront the data to send back in a scan response)? Or should the beacon listen for SCAN_REQ packets and broadcast a SCAN_RSP when it sees one?

I've been looking for libraries to use in Rust or Go, however support for developing a BLE peripheral seems to be lacking when using bluez in those languages.

I'm fine with answers in any programming language / library, as long as it works on Linux

The closes I've gotten so far is using bluer for Rust.

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let session = bluer::Session::new().await?;
    let adapter = session.default_adapter().await?;
    adapter.set_powered(true).await?;

    println!(
        "Advertising on Bluetooth adapter {} with address {}",
        adapter.name(),
        adapter.address().await?
    );

    let mut data = BTreeMap::new();
    data.insert(0, vec![1, 2, 3]);

    let le_advertisement = Advertisement {
        advertisement_type: bluer::adv::Type::Broadcast,
        local_name: Some("le_advertise".to_string()),
        advertisting_data: data,
        ..Default::default()
    };

    println!("{:?}", &le_advertisement);
    let handle = adapter.advertise(le_advertisement).await?;

    std::thread::sleep(std::time::Duration::from_secs(30));

    println!("Removing advertisement");
    drop(handle);
    Ok(())
}

This works for broadcasting an advertisement. I can see it on my phone using nRF Connect.

However I cannot find a way to respond to scan requests, nor a way to indicate that the beacon can be scanned.

Starting with the theoretical part, you are right about your understanding of the scan response. The official definition from the Bluetooth SIG is (Bluetooth Core Specification v5.4, Vol 1, Part A, section 4.2.2.2):-

> Advertising devices may receive scan requests from listening devices
> in order to get additional user data from the advertising device. Scan
> responses are sent by the advertising device to the device making the
> scan request.

And this is also mentioned here (Bluetooth Core Specification v5.4, Vol 1, Part A, section 3.3.2.2.2):-

> Some advertising events sent by the advertising device permit the
> listening device (scanner) to concurrently send scan requests or
> connection requests packets on the same advertising PHY channel in
> which the advertising packet was received. The advertising device can
> send a scan response packet again on the same advertising PHY channel
> within the same advertising event.

Scan responses are usually set before a device even starts advertising, however, it is permitted that the scan response is set or changed when the device has already started advertising. This is mentioned in the Bluetooth Specification v5.4, Vol 4, Part E, 7.8.8 LES Set Scan Response Data command:-

> If advertising is currently enabled, the Controller shall use the new
> data in subsequent advertising events. If an advertising event is in
> progress when this command is issued, the Controller may use the old
> or new data for that event. If advertising is currently disabled, the
> data shall be kept by the Controller and used once advertising is
> enabled.
>
> The default Scan_Response_Data_Length shall be zero and the default
> Scan_Response_Data shall be 31 zero octets.

As for the practical part, you can easily send scan reports using the btmgmt tool and the following command on Linux:-

sudo btmgmt add-adv -d 02010606084142434400 -s 080954657374204C45

Where the -d option is for setting the advert data, and the -s is for setting the scan response data. The full list of btmgmt add-adv options are:-

Usage: add-adv [options] <instance_id>

Options:
	 -u, --uuid <uuid>         Service UUID
	 -d, --adv-data <data>     Advertising Data bytes
	 -s, --scan-rsp <data>     Scan Response Data bytes
	 -t, --timeout <timeout>   Timeout in seconds
	 -D, --duration <duration> Duration in seconds
	 -P, --phy <phy>           Phy type, Specify 1M/2M/CODED
	 -c, --connectable         "connectable" flag
	 -g, --general-discov      "general-discoverable" flag
	 -l, --limited-discov      "limited-discoverable" flag
	 -n, --scan-rsp-local-name "local-name" flag
	 -a, --scan-rsp-appearance "appearance" flag
	 -m, --managed-flags       "managed-flags" flag
	 -p, --tx-power            "tx-power" flag
e.g.:
	add-adv -u 180d -u 180f -d 080954657374204C45 1

BLE data in adverts is decoded as follows (based on the Assigned Numbers Document):-

• 1st byte = length (n bytes)
• 2nd byte = Types
• n-1 bytes = actual data

So the meaning of the advert data I added:-

• 02 Length (2 bytes)

• 01 Type (Flags)

• 06 Flag - 02 && 04 LE General Discoverable && BR/EDR Not supported

• 06 Length (6 bytes)

• 08 Type (Shortened Local Name)

• 4142434400 (ABCD)

And the meaning of the scan response data is:-

• 08 Length (8 bytes)
• 09 Type (Complete Local Name)
• 54657374204C45 (Test LE)

Some more useful links:-

• Bluetooth 4.0 scan response

• btmgmt add-adv command not working

• BLE advertising raw data - how to decode

As you have said that you will accept answers using any programming language/library -- I'd like to point you to the Qt framework. In particular the Qt connectivity module and its bluetooth component. The ask is for a bluez implementation: on Linux, the Qt bluetooth module is a C++ wrapper around bluez. Here's an example excerpt in which both advertising data and scan response data are configured.

QLowEnergyAdvertisingData advertisingData;
QLowEnergyAdvertisingData scanResponseData; // <- added to original example
advertisingData.setDiscoverability(QLowEnergyAdvertisingData::DiscoverabilityGeneral);
advertisingData.setIncludePowerLevel(true);
advertisingData.setLocalName("HeartRateServer");
advertisingData.setServices(QList<QBluetoothUuid>() << QBluetoothUuid::ServiceClassUuid::HeartRate);
scanResponseData.setManufacturerData(0x1234, QByteArray("Hello"));  // <- added to example
bool errorOccurred = false;
const QScopedPointer<QLowEnergyController> leController(QLowEnergyController::createPeripheral());
auto errorHandler = [&leController,&errorOccurred](QLowEnergyController::Error errorCode)
{
        qWarning().noquote().nospace() << errorCode << " occurred: "
            << leController->errorString();
        if (errorCode != QLowEnergyController::RemoteHostClosedError) {
            qWarning("Heartrate-server quitting due to the error.");
            errorOccurred = true;
            QCoreApplication::quit();
        }
};
QObject::connect(leController.data(), &QLowEnergyController::errorOccurred, errorHandler);

QScopedPointer<QLowEnergyService> service(leController->addService(serviceData));
leController->startAdvertising(QLowEnergyAdvertisingParameters(), advertisingData,
                               scanResponseData);  // <- modified from original example

The above example illustrates how bluetooth stacks generally implement the scan response (specifically, note the second line and the last line in the example). Full source for the example is at https://code.qt.io/cgit/qt/qtconnectivity.git/tree/examples/bluetooth/heartrate-server/main.cpp?h=6.4 and is available under BSD license (note, however, the above excerpt was modified to highlight how to set the scan response).

I've verified the above code on RPi, and I can see that the resulting BTLE advertisement has the added mfr data using NRFConnect. To replicate my results on rpi:

1. sudo apt install qt5-default qtcreator qtconnectivity5-examples qtconnectivity5-dev
2. copy the examples to somewhere you can work on them (e.g. mkdir test && cd test && cp -R /usr/lib/arm-linux-gnueabihf/qt5/examples .) [I used an old rpi I had laying around -- yours may be newer and thus the examples will live in a different directory by default]
3. edit the example code (e.g. cd examples/bluetooth/heartrate-server && vim main.cpp) -- make the 3 edits I indicate above
4. qmake heartrate-server.pro
5. make
6. sudo ./heartrate-server

This example does not implement a beacon per se. However, the advertising portion of a ble device is largely the same whether the device is a beacon (advertising only) or if it has a service (or services). Connectability and the ability to respond to a scan request are orthogonal traits for a ble device. If you never connect to the toy service in the example, then it is functionally a beacon. In practice, commercial beacons do implement a service -- for the owner of the beacon to use when updating the info that the beacon is meant to convey to passers-by.
In any case, the above method would be employed when implementing a device that is strictly a beacon. Furthermore, you've indicated that you want to implement 'a beacon' but have given no further info to go on as to what type of beacon. Typically, one implements an iBeacon or an EddyStone beacon -- of course the BLE spec is open enough to allow you to invent your own beacon, too.

To answer your question about how the beacon implementation should behave: The scan response packet has the same structure as the advertisement packet, and it is normally configured at the same time that the advertisement data is configured. The Nordic stack, for example, works this way. Obviously, you can see above that's also how QT does it.

The raw linux HCI interface does it only slightly differently in that one must specify the scan response as a separate command, but the payload has the same format as the advertising data setup. Both the adv data and the scan response data are set up during the process of enabling advertising.

In general, your bluetooth package should enable you to set the scan response in a similar way to how you set the advertising data -- the two are quite closely linked. I'm much more familiar with the Nordic (softdevice) implementation, and on that particular platform one does have the option of fielding the SCAN_REQ event. That being said, the normal way to do things is to set up the scan response data ahead of time. From Linux userland, I doubt you can do anything else: the BTLE implementation needs to send the scan response pretty quickly after receiving the SCAN_REQ so there's no time for a round trip to userland. That data needs to be already in a buffer on the kernel side.

In your project, you need to do 2 things: First, indicate that your device can respond to the scan request. In the qt api, one does this with QLowEnergyAdvertisingParameters. Second, you need to provide the actual scan resonse data (see above).

Let's say you want to extend your rust package to provide the missing scan response function. One could do so by emulating the Qt implementation. If you dig around in the Qt implementation, you can (eventually) have a look at the implementation of QLeAdvertiserBluez; the method of interest is setScanResponseData which, in turn, delegates to setData. The call to setData is made with isScanResponsedata set to true, resulting in the OcfLeSetScanResponseData command being sent to the linux hci (along with the data in the object passed as the scan response). https://code.qt.io/cgit/qt/qtconnectivity.git/tree/src/bluetooth/qleadvertiser_bluez.cpp?h=5.15 line 339. OcfLeSetScanResponseData turns out to be opcode command field 0x9, which is passed to Qt's HciManager, and thence (via socket) to the linux HCI driver.

ocf 0x9 corresponds to

#define HCI_OP_LE_SET_SCAN_RSP_DATA	0x2009
struct hci_cp_le_set_scan_rsp_data {
	__u8	length;
	__u8	data[HCI_MAX_AD_LENGTH];

https://github.com/torvalds/linux/blob/master/include/net/bluetooth/hci.h Line 1651

So -- this is all doable. Qt obviously does it. What I can't figure out is why your Rust package did not expose this rather fundamental capability. And the BlueZ 'documentation' remains an impenetrable morass. Every time I think I want to use the userspace BlueZ stuff, I give up and just use the structure definitions from hci.h

[Edit]
While following up some dangling leads from the above research I found an alternate C API that may also be of interest. This one is direct from the bluez folks, but has a couple of major downsides. https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/src/shared That's bluez.git:src/shared -- apparently it is an unofficial API with no stability guarantees. Also, documentation for it appears to be thin to the point of non-existence (no worries, though -- code is self documenting, right?). There is an example at https://git.kernel.org/pub/scm/bluetooth/bluez.git/tree/tools/eddystone.c (bluez.git:tools/eddystone.c) which could probably be altered to make a beacon with a scan response. Look at adv_tx_power_callback in eddystone.c -- one could construct a bt_hci_cmd_le_set_scan_rsp_data struct, build the data array to contain a valid manufacturer info block (or a local name, or any other valid adv block), and then send the new struct with bt_hci_send(..., BT_HCI_CMD_LE_SET_SCAN_RSP_DATA,...)

[Final Edit]

Well, I've gotten pretty wrapped around this thing. But, for the sake of completeness: here is a full, working example of a pure (not connectable) eddystone beacon that also sends a response to the scan request. This, obviously is derived from the Qt example. It can be dropped into the above referenced heartrate server example, just replace main.cpp with this code.

In this example, I explicitly set the advertising parameters to indicate that the beacon is not connectable, but can respond to the scan request.

/***************************************************************************
**
** Copyright (C) 2017 The Qt Company Ltd.
** Contact: https://www.qt.io/licensing/
**
** This file is part of the examples of the QtBluetooth module of the Qt Toolkit.
**
** $QT_BEGIN_LICENSE:BSD$
** Commercial License Usage
** Licensees holding valid commercial Qt licenses may use this file in
** accordance with the commercial license agreement provided with the
** Software or, alternatively, in accordance with the terms contained in
** a written agreement between you and The Qt Company. For licensing terms
** and conditions see https://www.qt.io/terms-conditions. For further
** information use the contact form at https://www.qt.io/contact-us.
**
** BSD License Usage
** Alternatively, you may use this file under the terms of the BSD license
** as follows:
**
** "Redistribution and use in source and binary forms, with or without
** modification, are permitted provided that the following conditions are
** met:
**   * Redistributions of source code must retain the above copyright
**     notice, this list of conditions and the following disclaimer.
**   * Redistributions in binary form must reproduce the above copyright
**     notice, this list of conditions and the following disclaimer in
**     the documentation and/or other materials provided with the
**     distribution.
**   * Neither the name of The Qt Company Ltd nor the names of its
**     contributors may be used to endorse or promote products derived
**     from this software without specific prior written permission.
**
**
** THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
** "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
** LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
** A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
** OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
** SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
** LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
** DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
** THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
** (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
** OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE."
**
** $QT_END_LICENSE$
**
****************************************************************************/

#include <QtBluetooth/qlowenergyadvertisingdata.h>
#include <QtBluetooth/qlowenergyadvertisingparameters.h>
#include <QtBluetooth/qlowenergycharacteristic.h>
#include <QtBluetooth/qlowenergycharacteristicdata.h>
#include <QtBluetooth/qlowenergydescriptordata.h>
#include <QtBluetooth/qlowenergycontroller.h>
#include <QtBluetooth/qlowenergyservice.h>
#include <QtBluetooth/qlowenergyservicedata.h>
#include <QtCore/qbytearray.h>
#include <QtCore/qcoreapplication.h>
#include <QtCore/qlist.h>
#include <QtCore/qloggingcategory.h>
#include <QtCore/qscopedpointer.h>
#include <QtCore/qtimer.h>

int main(int argc, char *argv[])
{
    //QLoggingCategory::setFilterRules(QStringLiteral("qt.bluetooth* = true"));
    QCoreApplication app(argc, argv);

    //! [Advertising Data]
    QLowEnergyAdvertisingData advertisingData;
    QLowEnergyAdvertisingData scanRespData;

    uint8_t es2[21];  // eddystone advertising block -- taken from bluez eddystone example
    es2[0] = 0x02;   /* Field length */
    es2[1] = 0x01;   /* Flags */
    es2[2] = 0x04;   /* BR/EDR Not Supported */

    es2[3] = 0x03;   /* Field length */
    es2[4] = 0x03;   /* 16-bit Service UUID list */
    es2[5] = 0xaa;   /* Eddystone UUID */
    es2[6] = 0xfe;

    es2[7] = 0x0c;   /* Field length */
    es2[8] = 0x16;   /* 16-bit Service UUID data */
    es2[9] = 0xaa;   /* Eddystone UUID */
    es2[10] = 0xfe;
    es2[11] = 0x10;    /* Eddystone-URL frame type */
    es2[12] = 0x00;    /* Calibrated Tx power at 0m */
    es2[13] = 0x00;    /* URL Scheme Prefix http://www. */
    es2[14] = 'b';
    es2[15] = 'l';
    es2[16] = 'u';
    es2[17] = 'e';
    es2[18] = 'z';
    es2[19] = 0x01;    /* .org/ */

    es2[20] = 0x00;    /* Field terminator */
    advertisingData.setRawData(QByteArray((const char*)es2, 21));

    scanRespData.setManufacturerData(0x1234, QByteArray("hello"));
    //! [Advertising Data]

    //! [Start Advertising]
    const QScopedPointer<QLowEnergyController> leController(QLowEnergyController::createPeripheral());
    //const QScopedPointer<QLowEnergyService> service(leController->addService(serviceData));
    auto advParams = QLowEnergyAdvertisingParameters();
    advParams.setMode(QLowEnergyAdvertisingParameters::AdvScanInd);
    leController->startAdvertising(advParams, advertisingData,
                                   scanRespData /*advertisingData*/);
    //! [Start Advertising]


    auto reconnect = [&leController, advertisingData, scanRespData, advParams]() {
        leController->startAdvertising(advParams, advertisingData,
                                       scanRespData);
    };
    QObject::connect(leController.data(), &QLowEnergyController::disconnected, reconnect);

    return app.exec();
}