Skip to content

[SPARK-58620][EXAMPLES] Fix incorrect comments and stale usage docs in the custom receiver examples - #57822

Open
uros-b wants to merge 1 commit into
apache:masterfrom
uros-b:example-javacustomreceiver
Open

[SPARK-58620][EXAMPLES] Fix incorrect comments and stale usage docs in the custom receiver examples#57822
uros-b wants to merge 1 commit into
apache:masterfrom
uros-b:example-javacustomreceiver

Conversation

@uros-b

@uros-b uros-b commented Aug 6, 2026

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

Fixes documentation defects in the custom receiver examples:

  1. JavaCustomReceiver documents a <master> argument the example does not accept. The class comment is corrected to Usage: JavaCustomReceiver <hostname> <port> and the stale <master> explanation is dropped. The <hostname>/<port> line, which had no verb, is reworded to "describe the TCP server ...".
  2. The onStop() comment described the loop continuation condition instead of the stop condition. Corrected to "is designed to stop by itself when isStopped() returns true" in JavaCustomReceiver.java, CustomReceiver.scala, and both snippets in docs/streaming-custom-receivers.md.
  3. Received bytes is interpreted as becomes are in JavaCustomReceiver.java.

Why are the changes needed?

Each defect is verifiable against the code in the same file:

  • main() gates on args.length < 2 and reads only args[0] as hostname and args[1] as port; SparkConf is built with setAppName only, and setMaster has never appeared in this file. The in-code System.err.println already prints the two-argument usage, so only the class comment disagreed. The stale text is a leftover of SPARK-1826, which removed the positional master argument but left the comment block partly updated. This is the last remaining <master> usage text under examples/.
  • The receive loop is while (!isStopped() && ...) in both the Java and Scala examples, so the thread exits when isStopped() returns true. The old comment stated the opposite, and also lacked a conjunction. The docs page hand-inlines these snippets rather than using include_example, so it had drifted the same way and is fixed alongside.
  • "Received bytes" is plural. SPARK-15645 fixed this in CustomReceiver.scala but missed the Java counterpart.

The onStop() fix is applied to all four occurrences so the Java example, its Scala twin, and the docs snippets stay consistent.

Does this PR introduce any user-facing change?

No. Comments, example usage text, and documentation snippets only.

How was this patch tested?

No functional change; nothing executable is modified. The corrected statements were each checked against the surrounding code (argument parsing and the while (!isStopped()) loop).

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 4.8)

@uros-b uros-b left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Waiting for CI. @gengliangwang Please review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant