Extend syzkaller coverage

[borkmann]

As a project it would be great to review and extend syzkaller coverage (external contractor?) for the networking-related syscall and netlink UAPIs.

[wdebruij]

Do you have a concrete proposal, Daniel? Targeting specific syscalls that lack coverage? You mention netlink, for instance.

A one time contract to fix up a large surface area that lacks coverage, or on-going support (which I don't know how we would fund that).

The syzkaller syscall definitions (https://github.com/google/syzkaller/tree/master/sys/linux) do get updated. We could also ask of new feature developers that expand the ABI to expand syzkaller, as part of the normal netdev review process.

[jmberg]

We can ask, but since Google requires a CLA [1], we certainly can't demand and to me even asking is borderline.

[1] https://github.com/google/syzkaller/blob/master/docs/contributing.md

[wdebruij]

One concrete request from the TSC: expanding netlink coverage.

[wdebruij]

Core netlink code has good coverage by syzbot already. 83% of net/netlink. vs 52% for net/ as a whole. From https://syzkaller.appspot.com/upstream/coverage?period=month

See also the list of socket_netlink_*.txt files in https://github.com/google/syzkaller/tree/master/sys/linux for specific coverage.

Syzkaller team has also used static analysis [1] to generate netlink syzkaller signatures [2]. All around amazingly impressive work by the syzkaller team imho.

We should create a list specific areas where we believe coverage needs to be improved. Both concrete netlink coverage, if we think the above misses anything (else I'm now going to assume that that is well covered), and other Netdev areas. The above coverage dashboard will be a helpful starting point.

@kuba-moo FYI

[1] https://github.com/google/syzkaller/blob/master/tools/syz-declextract/README.md
[2] https://raw.githubusercontent.com/google/syzkaller/refs/heads/master/sys/linux/auto.txt

[kuba-moo]

All the static family definitions are a bit old, I don't see one for netdev family at all (and yet the file has impressive 75% coverage).
To propose something concrete I think we should try to add generator that will pop out the syzcaller .txt descriptions based on YAML specs?

[dvyukov]

To propose something concrete I think we should try to add generator that will pop out the syzcaller .txt descriptions based on YAML specs?

Hi! This would be useful.
The tool could read in YAML files where it reads in syscall *.tbl files from the kernel tree:
https://github.com/google/syzkaller/blob/96a211bca8bcc72de621f8990d476177d3463857/tools/syz-declextract/declextract.go#L347-L376

And then spew the same objects as created by the Clang tools from the source code analysis (probably replacing these):
https://github.com/google/syzkaller/blob/96a211bca8bcc72de621f8990d476177d3463857/pkg/declextract/entity.go#L117-L141

The rest should work automatically.

[a-nogikh]

To propose something concrete I think we should try to add generator that will pop out the syzcaller .txt descriptions based on YAML specs?

Given that syz-declextract can already extract netlink policies from the kernel (and that the yaml policies are internally converted to C policy tables anyway), I wonder if adding the support for the direct yaml -> syzlang conversion will help us get more kernel code coverage. Is there anything in the YAML policies that does not get translated to the C code?

All the static family definitions are a bit old, I don't see one for netdev family at all (and yet the file has impressive 75% coverage).

FWIW we have netdev stuff in the file with the auto-generated descriptions.

You could grep https://github.com/google/syzkaller/blob/master/sys/linux/auto.txt for syz_genetlink_get_family_id$auto_netdev and find the netdev policy descriptions below it.

[wdebruij]

That is great news. Thanks @a-nogikh!

So we can assume coverage for YNL generated netlink.

And can use the syzkaller coverage dash to find cases where coverage still falls short.

This may be due to APIs requiring specific net device types or other preconditions. See the current SO_DEVMEM work for a fairly complex example.

Let's review the dash specific to netlink and come up with more concrete items based on that.

[wdebruij]

Took a look at the syzkaller coverage dash.

I encourage everyone to do the same, in particular maintainers of individual subsystems.

In short, no silver bullets.

Something like devmem is expected. Requires work (ongoing, non-trivial) for syzkaller to exercise the code. The same is true, for instance, for device driver tso in net/core/tso.c.

IPSec and TLS are perhaps in the same category.

If anything, the largest surprising surface to me is the moderate coverage of netfilter. All those modules syzkaller should be able to reach. Some bpf code is in the same space category, though harder as the BPF code needs to be generated.

The various sysctl files have decent coverage, but also surprising that custom sysctl setters for RFS or flow limit are not covered. In absolute terms it's not a significant gap.

• netfilter
• devmem
• netdev_rx_queue
• netpoll
• sysctl.c
• ovs
• ipsec, tls
• bpf