Rate-limit unauthenticated crawling of profile solutions (#9421)

api/profiles/solutions#index is enumerable - one URL per user handle, and
handles are public - unauthenticated, and cheap to request. It had no GET
throttle at all, so it belongs in the existing crawl throttle alongside the
exercise pages and submission files.

Found while breaking down CloudWatch log ingestion: a single netcup VPS
(2a0a:4cc0:2000:2e9b:1434:8eff:feda:ee1) has been walking the handle space at
a flat ~1,250 req/hr around the clock - 30,075 404s to 68 200s in 24 hours,
roughly 900k requests/month. Nothing stopped it. Cloudflare's "60 in 1m" rule
is scoped `http.host eq "exercism.org"` and this traffic arrives on
api.exercism.org, so that rule never evaluates.

Fixing it here rather than at the edge covers both hostnames at once and
survives the next IP. Note that api.exercism.org cannot simply be put behind
the same bot challenge as exercism.org, because the CLI talks to
api.exercism.io/v1.


Claude-Session: https://claude.ai/code/session_01WRm6FbLza51hYVjkTSuGju

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

Author: iHiD

config/initializers/rack_attack.rb

@@ -70,21 +70,22 @@ def signed_in?
   req.env['HTTP_X_SEARCH_ENGINE'] == 'true'
 end
 
-# These two endpoints are cheap to request and expensive to serve, and are
-# enumerable: there is one URL per exercise and one per submission, so anything
-# walking the site systematically can pull an unbounded number of them. Cap
-# unauthenticated access per-IP per-day. The limit is far above what any
-# person browsing the site would reach.
+# These endpoints are cheap to request and expensive to serve, and are
+# enumerable: there is one URL per exercise, one per submission, and one per
+# user handle, so anything walking the site systematically can pull an
+# unbounded number of them. Cap unauthenticated access per-IP per-day. The
+# limit is far above what any person browsing the site would reach.
 CRAWLED_EXERCISE_PATH = %r{\A/tracks/[^/]+/exercises/[^/]+(/|\z)}
 Rack::Attack.throttle("Unauthenticated crawling of expensive endpoints", limit: 500, period: 1.day) do |req|
   next unless req.get?
   next if req.signed_in?
 
-  # Only resolve the route for the API endpoint, and only once we know the
+  # Only resolve the route for the API endpoints, and only once we know the
   # path is in the right namespace: recognize_path walks the whole route set,
   # which is far too expensive to pay on every unauthenticated GET.
   matches = req.path.match?(CRAWLED_EXERCISE_PATH) ||
-            (req.path.starts_with?('/api/v2/solutions') && req.routed_to == 'api/solutions/submission_files#index')
+            (req.path.starts_with?('/api/v2/solutions') && req.routed_to == 'api/solutions/submission_files#index') ||
+            (req.path.starts_with?('/api/v2/profiles') && req.routed_to == 'api/profiles/solutions#index')
   next unless matches
 
   "unauthenticated-crawl|#{req.client_ip}"

test/initializers/rack_attack_test.rb

@@ -224,6 +224,33 @@ class RackAttackTest < Webhooks::BaseTestCase
     end
   end
 
+  test "unauthenticated profile solutions requests over the limit are throttled" do
+    profile_user = create(:user_profile).user
+    ip = "1.2.3.20"
+
+    freeze_time do
+      fill_crawl_throttle(ip, 500)
+
+      get api_profile_solutions_path(profile_user), headers: crawler_headers(ip)
+      assert_response :too_many_requests
+    end
+  end
+
+  # Enumerating the handle space mostly produces 404s. Rack::Attack is
+  # middleware, so it counts the request before the app decides the profile
+  # doesn't exist — which is the whole point here, since a miss is exactly what
+  # a crawler walking usernames generates.
+  test "profile solutions requests for handles that do not exist still count towards the crawl throttle" do
+    ip = "1.2.3.21"
+
+    freeze_time do
+      fill_crawl_throttle(ip, 500)
+
+      get api_profile_solutions_path("no-such-handle"), headers: crawler_headers(ip)
+      assert_response :too_many_requests
+    end
+  end
+
   test "authenticated requests are not throttled by the crawl throttle" do
     create :practice_exercise
     ip = "1.2.3.7"