Skip to content

feat: reattach named child runs after migration or resurrection - #1149

Open
vdusek wants to merge 7 commits into
masterfrom
feat/named-child-runs
Open

vdusek wants to merge 7 commits into
masterfrom
feat/named-child-runs

Conversation

@vdusek

@vdusek vdusek commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Actor.start, Actor.call and Actor.call_task accept an optional run_name. The SDK writes the name, the Actor or task ID and the run ID to the default key-value store (APIFY_CHILD_RUNS) right after the child starts. After a migration or resurrection of the parent, the same call finds the recorded run and reuses it:

  • READY / RUNNING: reattaches to it
  • SUCCEEDED: returns it as is
  • ABORTED / TIMED-OUT: resurrects it (an ABORTING / TIMING-OUT run is waited for first)
  • FAILED, or the run no longer exists: starts a new run and keeps the old ID in previousRunIds

A named call on a reattached or resurrected run streams only new log lines. Reusing a name for a different Actor or task raises ValueError, and concurrent calls under one name in the same process share a single run.

A hard kill between the platform starting the child and the registry write can still orphan the child. Closing that gap needs an idempotency key on the run-start endpoint.

Closes: #1127

✍️ Drafted by Claude Code

@vdusek vdusek added the t-tooling Issues with this label are in the ownership of the tooling team. label Sep 29, 2026
@vdusek vdusek self-assigned this Sep 29, 2026
@codecov

codecov Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.12%. Comparing base (32a0a30) to head (ed8160c).

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #1149       +/-   ##
===========================================
+ Coverage   53.03%   93.12%   +40.09%     
===========================================
  Files          53       54        +1     
  Lines        3560     3665      +105     
===========================================
+ Hits         1888     3413     +1525     
+ Misses       1672      252     -1420     
Flag Coverage Δ
e2e 32.82% <39.44%> (+0.21%) ⬆️
integration 52.68% <39.44%> (-0.35%) ⬇️
unit 89.82% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek
vdusek added this pull request to stack #1152 September 30, 2026 07:49
@vdusek
vdusek force-pushed the feat/named-child-runs branch from 82366a3 to 8e76335 Compare October 6, 2026 13:20
@vdusek
vdusek marked this pull request as ready for review October 6, 2026 13:33
@apify-service-account apify-service-account added the tested Temporary label used only programatically for some analytics. label Oct 6, 2026
@vdusek
vdusek requested review from Mantisus and barjin October 7, 2026 06:10
@vdusek vdusek mentioned this pull request Oct 7, 2026
@vdusek

vdusek commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor Author

TODO:

Both need master merged into the stack first.

✍️ Drafted by Claude Code

Comment thread src/apify/_child_runs.py
Comment on lines +120 to +121
run_client = client.run(record.run_id)
run = await run_client.get()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_wait_for_child_run waits on client.run(...) through wait_for_finish(), which tolerates 404s for a few seconds after a run starts. Here we use a bare get(), and a 404 from it starts a new run. Is that on purpose?

Comment thread src/apify/_child_runs.py
Comment on lines +72 to +73
self._load_lock = asyncio.Lock()
self._write_lock = asyncio.Lock()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we use only one for both operations? 🙂

Comment thread src/apify/_child_runs.py
self._records: dict[str, ChildRunRecord] | None = None
self._load_lock = asyncio.Lock()
self._write_lock = asyncio.Lock()
self._name_locks: defaultdict[str, asyncio.Lock] = defaultdict(asyncio.Lock)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can use WeakValueDictionary here

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Named child runs that survive migration and resurrection

3 participants