-
Notifications
You must be signed in to change notification settings - Fork 1.4k
feat(domain-skills): add mastodon engagement skill #369
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
drmweyers
wants to merge
5
commits into
browser-use:main
Choose a base branch
from
drmweyers:feat/mastodon-skill
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
46ecb93
feat(domain-skills): add Mastodon engagement skill
drmweyers 3dfe529
Merge remote-tracking branch 'upstream/main' into feat/mastodon-skill
drmweyers d6f6759
skill(instagram): engagement selector playbook (like/follow/comment/dm)
drmweyers 6084100
skill(tiktok): engagement selector playbook (like/follow/comment)
drmweyers 1d50d56
skill(facebook): engagement selector playbook (like + comment only)
drmweyers File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| # Facebook — Engagement (like · comment) | ||
|
|
||
| Selector playbook for **engaging** on facebook.com web — **like and comment only**. | ||
| This brand acts as a **dedicated personal profile** (not a Page). **DM (Messenger) | ||
| and follow/friend are intentionally NOT supported** — they are the highest Meta-ban | ||
| actions and fail closed. | ||
|
|
||
| > This file documents the DOM contract the SmartSocial stagehand executor pins to | ||
| > (`agents/bob/runtime/stagehand-executor.ts`): identity-first, fixed selector | ||
| > allowlist, editable/clickable verification, block scan, fail-closed. Facebook | ||
| > obfuscates its DOM heavily and rotates class names constantly — these are | ||
| > **supervised-first**: confirm every read against the live UI before unsupervised | ||
| > runs, and expect to re-verify after Facebook redesigns. | ||
|
|
||
| ## Identity (assert BEFORE any action) | ||
|
|
||
| Facebook does not expose a clean handle in the chrome. The most stable logged-in | ||
| signal is the **`c_user` cookie** — the numeric profile id, not httpOnly. So set | ||
| the brand's `account_handle` to that numeric id. Fallback: the top-nav "Your | ||
| profile" shortcut's vanity username. Returns `""` (fail closed) if not logged in: | ||
|
|
||
| ```js | ||
| (() => { | ||
| const m = document.cookie.match(/(?:^|;\s*)c_user=(\d+)/); | ||
| if (m) return m[1]; // numeric profile id (preferred) | ||
| const a = document.querySelector('a[aria-label="Your profile"], [aria-label="Your profile"] a[href]'); | ||
| const href = a ? a.getAttribute("href") : null; | ||
| const um = href ? href.match(/facebook\.com\/([A-Za-z0-9.]+)\/?(?:$|\?)/) : null; | ||
| return um ? um[1] : ""; // vanity username fallback | ||
| })() | ||
| ``` | ||
|
|
||
| ## Selectors | ||
|
|
||
| | Action | Element | Selector / signal | | ||
| |---|---|---| | ||
| | **Like** | post Like control | `div[aria-label="Like"][role="button"]` — aria-label flips to `Remove Like` once liked | | ||
| | **Comment** | comment composer | `div[contenteditable="true"][role="textbox"]` (aria-label `Write a comment…`) | | ||
| | **Comment submit** | — | press `Enter` in the composer | | ||
| | **DM / follow / friend** | — | **unsupported** (fail closed) | | ||
|
|
||
| ### Like — click-toggle (read `aria-label`) | ||
|
|
||
| Facebook's Like button has no reliable `aria-pressed`; the signal is the button's | ||
| own `aria-label` flipping `Like` → `Remove Like`. **Idempotency: if it already reads | ||
| `Remove Like`, DO NOT click — a second click un-likes (or opens the reaction | ||
| picker).** Click triggers a plain Like (don't hover — hovering opens Love/Haha/etc). | ||
|
|
||
| ```python | ||
| js(r'''(() => { | ||
| const b = document.querySelector('div[aria-label="Like"][role="button"], div[aria-label="Remove Like"][role="button"]'); | ||
| if (!b) return null; | ||
| const r = b.getBoundingClientRect(); | ||
| return JSON.stringify({x: Math.round(r.x+r.width/2), y: Math.round(r.y+r.height/2), | ||
| label: b.getAttribute("aria-label")}); | ||
| })()''') | ||
| # if label === "Remove Like": already liked → done, no click. | ||
| # else click_at_xy(...) then re-read: success when label becomes "Remove Like". | ||
| ``` | ||
|
|
||
| ### Comment — text-input | ||
|
|
||
| ```python | ||
| js('document.querySelector(\'div[contenteditable="true"][role="textbox"]\').focus()') | ||
| type_text("your comment here") | ||
| press_key("Enter") | ||
| ``` | ||
|
|
||
| ## Notes / risk | ||
|
|
||
| - **Meta ban risk is real.** Use a warmed, dedicated brand profile on a residential | ||
| IP, conservative caps, slow ramp — the same posture as Instagram. A flagged | ||
| profile can take the linked Instagram account down too. | ||
| - Block / checkpoint markers ("You're Temporarily Blocked", captcha) and any | ||
| redirect to `/login` or `/checkpoint/` map to fail-closed outcomes upstream. | ||
| - Facebook redesigns frequently — treat these selectors as a starting point to | ||
| re-confirm, not a guarantee. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| # Instagram — Engagement (like · follow · comment · DM) | ||
|
|
||
| Selector playbook for **engaging** on instagram.com web (not posting). Attach to a | ||
| Chrome profile already logged into the target account. Two action *shapes*: | ||
|
|
||
| - **Text-input shape** (comment, DM): focus an editable input → type → submit. | ||
| - **Click-toggle shape** (like, follow): click a button whose `aria-label`/text | ||
| flips state after the action (`Like`→`Unlike`, `Follow`→`Following`). | ||
|
|
||
| > This file documents the same DOM contract the SmartSocial stagehand executor | ||
| > pins to (`agents/bob/runtime/stagehand-executor.ts`): identity-first, fixed | ||
| > selector allowlist, editable/clickable verification, block scan, fail-closed. | ||
|
|
||
| ## Identity (assert BEFORE any action) | ||
|
|
||
| The logged-in handle is read from the **session chrome**, never from page text | ||
| (a post comment containing the handle could spoof an LLM read). Left-nav own-profile | ||
| entry renders the session avatar: | ||
|
|
||
| ```js | ||
| // returns the logged-in handle, or "" (fail closed) | ||
| (() => { | ||
| const scope = document.querySelector('div[role="navigation"], nav') || document; | ||
| const img = scope.querySelector('img[alt$="profile picture"]'); | ||
| if (!img) return ""; | ||
| const a = img.closest('a[href^="/"]'); | ||
| const m = a && (a.getAttribute("href") || "").match(/^\/([A-Za-z0-9._]+)\/?$/); | ||
| if (m) return m[1]; | ||
| const am = (img.getAttribute("alt") || "").match(/^(.+?)'s profile picture$/); | ||
| return am ? am[1] : ""; | ||
| })() | ||
| ``` | ||
|
|
||
| If this returns `""` → treat as logged-out, do nothing. | ||
|
|
||
| ## Selectors | ||
|
|
||
| | Action | Element | Selector / signal | | ||
| |---|---|---| | ||
| | **Like** | heart button on a post | `svg[aria-label="Like"]` (liked → `svg[aria-label="Unlike"]`); click the enclosing `div[role="button"]`/`button` | | ||
| | **Comment** | comment input under a post | `textarea[aria-label="Add a comment…"]` (note the ellipsis char `…`, U+2026) | | ||
| | **Comment submit** | Post button | adjacent `[role="button"]` with text `Post`, or press `Enter` in the textarea | | ||
| | **Follow** | profile-header button | `button` whose trimmed text is `Follow` (followed → `Following` / `Requested`) | | ||
| | **DM input** | conversation composer | `div[role="textbox"][contenteditable="true"]` (aria-label `Message`) | | ||
| | **DM send** | — | press `Enter` in the composer | | ||
|
|
||
| ### Like — click-toggle | ||
|
|
||
| ```python | ||
| # Verify it's a real button, not a link/report-flow, then click the heart. | ||
| # Success = aria-label flips Like -> Unlike (read AFTER the click). | ||
| js(r'''(() => { | ||
| const svg = document.querySelector('svg[aria-label="Like"], svg[aria-label="Unlike"]'); | ||
| if (!svg) return null; | ||
| const btn = svg.closest('div[role="button"], button'); | ||
| if (!btn) return null; | ||
| const r = btn.getBoundingClientRect(); | ||
| return JSON.stringify({x: Math.round(r.x+r.width/2), y: Math.round(r.y+r.height/2), | ||
| state: svg.getAttribute("aria-label")}); | ||
| })()''') | ||
| # click_at_xy(...) then re-read: state must now be "Unlike" | ||
| ``` | ||
|
|
||
| ### Follow — click-toggle | ||
|
|
||
| ```python | ||
| # Click the header Follow button. Success = text becomes "Following" or "Requested" | ||
| # (private accounts return "Requested"; treat both as success). | ||
| js(r'''(() => { | ||
| const b = [...document.querySelectorAll('button')] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P1: Follow snippet performs an unscoped global button search and may click the wrong Follow button, such as suggested accounts or side rails, instead of the intended profile-header button. Prompt for AI agents |
||
| .find(b => ["Follow"].includes((b.textContent||"").trim())); | ||
| if (!b) return null; | ||
| const r = b.getBoundingClientRect(); | ||
| return JSON.stringify({x: Math.round(r.x+r.width/2), y: Math.round(r.y+r.height/2)}); | ||
| })()''') | ||
| ``` | ||
|
|
||
| ### Comment — text-input | ||
|
|
||
| ```python | ||
| js('document.querySelector(\'textarea[aria-label="Add a comment…"]\').focus()') | ||
| type_text("your brand-voice comment") | ||
| press_key("Enter") # or click the adjacent "Post" role=button | ||
| # Success = the comment text appears in the page AND was not pre-existing. | ||
| ``` | ||
|
|
||
| ## Block / safety markers (any present → stop, do not retry) | ||
|
|
||
| Soft-block toast text (case-insensitive): `Action Blocked`, `Try Again Later`, | ||
| `We limit how often`, `restrict certain activity`. | ||
|
|
||
| URL fragments that mean "not on target / logged out": `/accounts/login`, `/login`, | ||
| `/challenge/`, `/checkpoint/`. | ||
|
|
||
| ## Gotchas | ||
|
|
||
| - The comment textarea aria-label uses a real **ellipsis** `…` (U+2026), not three dots. | ||
| - **Like is an SVG inside a button** — verify `div[role="button"]`/`button`, never click the bare `<svg>` or a wrapping `<a>` (could be a report/profile link). | ||
| - A second un-acted **`Like` may exist** (e.g. nested reels/suggested) — pin to the first under the target post container; the stagehand observe instruction scopes "under this post". | ||
| - **Follow/unfollow churn is a known IG flag.** No auto-unfollow. Keep follow conservative. | ||
| - `Following` vs `Requested`: private accounts return `Requested` — both are success. | ||
| - Read like/follow state **after** the click for the post-condition; reading before proves nothing (idempotent re-runs would false-positive). | ||
| - All engagement is human-paced + rate-capped + (during ramp) approval-gated upstream; this file is selectors only. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,130 @@ | ||
| # Mastodon — Hashtag Discovery | ||
|
|
||
| Field-tested against `mastodon.social`, `fosstodon.org`, and `hachyderm.io` on 2026-05-15. | ||
| Vouched by SmartSocial — used in production by Bob (https://github.com/drmweyers/SmartSocial/tree/main/agents/bob). | ||
|
|
||
| **Federation caveat.** Mastodon is not one site. Every instance ships its own (server-side rendered) theme on top of the same `tootsuite/mastodon` upstream, so DOM hashes and CSS class names diverge between instances. Anchor on the **semantic class names** (`.status`, `.account`, `.detailed-status`) and `data-*` attributes that the upstream Rails app emits — those are stable across instances. Avoid layout-shell selectors (sidebar widths, navigation chrome) — those are themed per-instance. | ||
|
|
||
| ## URL patterns | ||
|
|
||
| | What | URL | | ||
| |------|-----| | ||
| | Hashtag timeline (federated) | `https://<instance>/tags/<tag>` | | ||
| | Hashtag timeline (local-only) | `https://<instance>/tags/<tag>?local=true` | | ||
| | Single status (canonical, instance-local) | `https://<instance>/@<user>/<status_id>` | | ||
| | Remote status (forwards through home instance) | `https://<instance>/@<user>@<remote_instance>/<status_id>` | | ||
| | Public local timeline | `https://<instance>/public/local` | | ||
| | Account profile | `https://<instance>/@<user>` | | ||
|
|
||
| `<tag>` is the hashtag **without** the `#`. `#climate` → `/tags/climate`. | ||
|
|
||
| ## DOM anchors | ||
|
|
||
| | Target | Selector | Notes | | ||
| |---|---|---| | ||
| | Status container | `article.status, article.detailed-status` | Detailed = the focused status on a permalink page. Both carry `data-id="<status_id>"`. | | ||
| | Status text body | `.status__content, .e-content` | Server-rendered HTML — `<p>` paragraphs, `<a class="mention">`, `<a class="hashtag">`. Read `.innerText` for plain text, or `.innerHTML` if you need links/mentions. | | ||
| | Author handle | `.display-name__account` | Always the federated form (`@user@instance.tld`). Even on the home instance, this is rendered with the full handle once the status is from a remote actor. | | ||
| | Author display name | `.display-name__html` | May contain custom emojis as `<img class="emojione">` — strip if you need plain text. | | ||
| | Status timestamp | `time.formatted-date` | The `datetime` attribute is an ISO 8601 string; the visible text is a relative-time renderer (e.g. "3h"). Always read `datetime`, never `innerText`. | | ||
| | Status permalink | `a.status__relative-time` | `href` is the canonical URL of the status on its origin instance. | | ||
| | Reply button | `button.status__action-bar__button.icon-button[aria-label="Reply"]` | aria-label is localized — match by `title` or by position in `.status__action-bar` (first icon button) if the user's locale isn't English. | | ||
| | Favourite button | `button.icon-button.star-icon[aria-label="Favourite"]` | Aria-pressed flips to `true` after click. | | ||
| | Boost button | `button.icon-button.reblog-icon[aria-label="Boost"]` | Same pattern as favourite. Do not boost as part of automated engagement — boosts amplify into your followers' timelines and look spammy. | | ||
| | Bot badge | `.display-name .bot, .account__header__bot` | Present when `account.bot === true`. **Skip these accounts.** See gotcha below. | | ||
|
|
||
| ## Bot accounts | ||
|
|
||
| **Skip statuses authored by accounts flagged as bots.** Mastodon convention is that bot operators set `bot: true` on their account; engaging back creates bot-on-bot loops that the community considers low-quality. The UI surfaces this as a small "BOT" badge next to the display name (`.account__header__bot` on profile pages, `.display-name .bot` in timelines). | ||
|
|
||
| ```python | ||
| is_bot = js("""(() => { | ||
| const el = document.querySelector('article.status .display-name .bot'); | ||
| return !!el; | ||
| })()""") | ||
| if is_bot: | ||
| # skip — do not extract this status for engagement | ||
| pass | ||
| ``` | ||
|
|
||
| ## Pulling a hashtag timeline | ||
|
|
||
| The federated timeline at `/tags/<tag>` is the canonical discovery surface — it includes posts from the entire fediverse that the home instance has cached, not just locals. The local-only variant (`?local=true`) is fine for niche instances (`fosstodon.org/tags/python`) where you want only that community's voice. | ||
|
|
||
| ```python | ||
| new_tab("https://mastodon.social/tags/climate") | ||
| wait_for_load() | ||
| wait(1.5) # SSR is fast but action-bar buttons hydrate after first paint | ||
|
|
||
| statuses = js(r""" | ||
| Array.from(document.querySelectorAll('article.status')).map(el => { | ||
| const time = el.querySelector('time.formatted-date'); | ||
| const link = el.querySelector('a.status__relative-time'); | ||
| const body = el.querySelector('.status__content'); | ||
| const acct = el.querySelector('.display-name__account'); | ||
| const isBot = !!el.querySelector('.display-name .bot'); | ||
| return { | ||
| status_id: el.getAttribute('data-id'), | ||
| url: link?.href || null, | ||
| author: acct?.innerText?.trim() || null, | ||
| created_at: time?.getAttribute('datetime') || null, | ||
| text: body?.innerText?.trim() || '', | ||
| is_bot: isBot, | ||
| }; | ||
| }).filter(s => s.status_id && !s.is_bot) | ||
| """) | ||
| ``` | ||
|
|
||
| ## Lazy load — scroll, don't paginate | ||
|
|
||
| The timeline is an infinite scroll. There is no "Next page" control. The DOM keeps mounted statuses (~50 visible at a time) and prepends new ones at the top via a small "Show new toots" banner when polling discovers them. | ||
|
|
||
| ```python | ||
| # Collect up to TARGET statuses by scrolling | ||
| TARGET = 30 | ||
| seen = {} | ||
| for _ in range(10): # cap scrolls | ||
| batch = js(...) # the JS block above | ||
| for s in batch: | ||
| seen.setdefault(s["status_id"], s) | ||
| if len(seen) >= TARGET: | ||
| break | ||
| scroll(640, 400, dy=900) | ||
| wait(1.0) | ||
| ``` | ||
|
|
||
| `wait(1.0)` is enough on `mastodon.social` and `fosstodon.org`. On smaller instances with slower hardware (single VPS), bump to `wait(2.0)` if the batch returns the same IDs twice. | ||
|
|
||
| ## Login wall on remote-status pages | ||
|
|
||
| When you click a status that originated on a **different** instance from the one you're browsing, Mastodon may render a federated-actor stub page rather than the full thread. If you are logged in, a "Sign in to participate" interstitial does **not** appear — the page renders with the action bar enabled. If you are anonymous, every action button is replaced with a "You need to be logged in" tooltip and clicking does nothing. | ||
|
|
||
| Detect with: | ||
|
|
||
| ```python | ||
| auth_wall = js(""" | ||
| (() => { | ||
| const interstitial = document.querySelector('.sign-in-banner, .columns-area__panels__main .activity-stream-tabs'); | ||
| const replyBtn = document.querySelector('button[aria-label="Reply"]:not([disabled])'); | ||
| return !!interstitial && !replyBtn; | ||
| })() | ||
| """) | ||
| if auth_wall: | ||
| # stop and ask the user to log in | ||
| pass | ||
| ``` | ||
|
|
||
| ## Rate-limit signals | ||
|
|
||
| Per the upstream Mastodon REST contract, anonymous browse has soft per-IP limits (varies by instance, typically ~300 requests / 5min). The web UI surfaces overage as a generic red toast: "Failed to fetch — please try again." Screenshot to verify; the toast lives in `.notification-bar` and auto-dismisses after 5s. | ||
|
|
||
| Don't try to "fix" this with retries — back off and revisit the hashtag in 5-10 minutes. Browsing the same hashtag every <60s for an extended period is a heuristic the operator may flag. | ||
|
|
||
| ## Gotchas | ||
|
|
||
| - **Federation lag.** A status posted on `fosstodon.org` 30 seconds ago may not appear in `mastodon.social/tags/python` for another 30-90 seconds. If you are polling for fresh content, query the origin instance directly when you know it. | ||
| - **`time.formatted-date.innerText` is relative**, not ISO. Always read the `datetime` attribute for sortable timestamps. | ||
| - **Hashtag is case-insensitive in the URL but case-preserved in the DOM.** `/tags/Python` and `/tags/python` resolve to the same timeline; matched-tag text in the status body keeps the author's original casing. | ||
| - **Custom emoji in display names.** `.display-name__html` may render `:emoji_shortcode:` as `<img class="emojione">`. Strip these for plain-text author labels. | ||
| - **Detailed-status pages have a different selector.** The permalink page (`/@user/status_id`) uses `article.detailed-status` for the focused status and `article.status` for the surrounding thread context. Match both with `article.status, article.detailed-status` if you want to scrape a thread. | ||
| - **Don't boost from automation.** The boost button works the same as favourite mechanically, but boosting amplifies into your own followers' timelines — it's a different social contract. Stick to reply + favourite for engagement workflows. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2: PR scope mismatch: an Instagram engagement skill file is added in a PR titled/described as adding only Mastodon domain skills.
Prompt for AI agents