Collective base

Protection → Collective base (collective, a function of the subscription; beta: open to every installation; internal/collective) is off by default on every box: an administrator (the permission rules.write) turns it on, once the terms of use are accepted. It asks the analysis service of Warda (warda-analyst, behind the online service) its opinion on the names the box doubts, and gives it in return the decisions of the administrators. Nothing is ever blocked by it alone.

What is sent, and only while it is on: registrable domains only (the domain bought, eTLD+1 of the Public Suffix List: evil-shop.com for a.b.evil-shop.com, lower case):

• the decisions of the administrators on the suggestions: accepted or rejected, with the role of the learning and its score (none for a suggestion of the collective base itself);
• the doubts of the box, with their reason: observed (a name under observation of the learning), lookalike (a misleading name told), young (a young domain seen), beacon (a beacon suspected), blocked_request (an account asked to unblock it, see Unblock requests); a doubt is sent again at most once a week while no opinion on it is kept, and at the next exchange while the analysis service has not answered it yet (pending in its answer).

Never sent: a device, a person, a time of query, an address, the history, the account that decided. The call is signed by the key of the box like its other calls to warda-cloud; the online service never passes the identity of the box to the analysis service, only a pseudonym of the month.

Rhythm: the first exchange 2 to 5 minutes after it is turned on (or after a start when the last one is old), then every 6 hours, or the delay the online service asks (next_after, 1 hour to 7 days); when decisions or doubts wait, they go together a minute after the first one, at most once an hour. At most 200 decisions and 200 doubts an exchange (the rest waits for the next one), at most 1000 opinions taken from an answer. After a failure: 5 minutes, then three times longer each time, up to 6 hours (the Retry-After of the service when longer, up to a day). Exchange now (POST /api/v1/collective/exchange) waits 5 minutes after the last exchange (409 when it is off, runs or ended less than 5 minutes ago). The task Collective base (collective) of Administration → Scheduled tasks shows them.

What is received: the opinion of the analysis service on domains — clean, suspicious, malicious or unknown, a score, a category (malware, phishing, scam, tracking, ads, spyware, control of infected devices, other), its reasons (10 at most), its date and its source (analyst, or manual: the Warda team decided) — for the doubts of the box and for the changes of the living list of the service. Each is kept for its lifetime (ttl: 24 hours when not said, 30 days at most). They show on the suggestion cards and the watched names ("Opinion of Warda: suspicious — registered 3 days ago, listed by …"), and the page lists them, searchable and filtered by opinion and source. When the analysis service is unavailable, the online service still answers (analyst: unavailable): the decisions and doubts stay waiting and go again at the next exchange (the service asks it within the hour).

Suggestions: a domain judged suspicious or malicious that the network asked within the last 7 days becomes a suggestion marked Collective base (at most 5 over 7 days, a cap of its own beside that of the learning; never a name a rule of the network already decides). It waits for an administrator like the others; accepted or rejected, it is a decision sent back.

The page says what is exchanged (sent, never sent, received, when), the switch Exchange with the collective base, the state (last exchange and whether it went through, next exchange, analysis service available or not), the counts of the last 7 days (decisions and doubts sent, opinions received, waiting to be sent, waiting for the analysis) and the opinions received. The dashboard card Collective base (a module, shown by default) gives the state, the last send, the last opinions received, the counts over 7 days and the state of the analysis service; off, it says so with a link Turn it on for the accounts that may. It is shown (and offered among the modules to add) only to the accounts that read the rules (rules.read). The tile Online services shows its real state. Turning it off stops the exchanges and forgets what waited to be sent (the opinions kept stay until their end, not used while it is off). The setting is in the backups; the opinions and the queue are not. The administration log writes collective.on, collective.off and collective.exchange (an exchange asked by a person).

API (rules.read to read, rules.write to change): GET /api/v1/collective (enabled, allowed: the terms in force accepted and the plan has the function, available, running, last, error, last_ok, last_sent, last_received, analyst, awaiting, next, counts, per_week); PUT /api/v1/collective {"enabled": true}; POST /api/v1/collective/exchange; GET /api/v1/collective/verdicts?verdict=&source=&q=. The suggestions carry a field collective (the opinion kept for their domain).