[pull] master from ruby:master#1213
Merged
Merged
Conversation
The first version of the inliner used the "high water mark" of HIR instructions ever allocated. This PR estimates the number of live instructions that we care about in `infer_types`. When inlining, it also bumps the estimate by the delta of new HIR instructions from inlining the callee. `infer_types` is somewhat arbitrarily chosen because it runs frequently between other passes and I don't want a whole other pass. Result: slightly different inlining behavior. On Lobste.rs, we go from ~5400 inlining rejected due to budget down to ~3900.
When gc_move relocates an object to a different slot size, rb_gc_obj_changed_slot_size may re-embed it: the fields move from the fields_obj into the object itself and the object references them directly. The write-barrier history for those (possibly young) values lived on the discarded fields_obj, so an old unremembered object then violates the O->Y invariant, which gc_verify_internal_consistency reports as a WB miss. Remember the object on such moves. Remembering inside rb_gc_obj_changed_slot_size (as #17866 tried) does not work for this path: gc_move calls it before the destination's age bits are restored, so RVALUE_OLD_P is false there, and gc_move overwrites the destination's remembered bits afterwards anyway. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ZJIT: Fold defined?(yield) when inlining method with blockiseq We know this statically. * ZJIT: Fold negative case of defined?(yield) too Leave handling non-blockiseq arguments for later (right now they should never appear in SendDirect instructions).
It tells you total size per symbol, average size per instance of symbol, and total count per symbol. Can be used with `--zjit-perf=hir`: ``` plum% ruby ../tool/zjit_analyze_perf_map_size.rb /tmp/perf-16653.map 12.1 MB total; 277.5 B each side-exit 2.2 MB total; 16.9 B each PatchPoint 2.1 MB total; 197.4 B each SendDirect 1.3 MB total; 10.7 B each PadPatchPoint 973.6 KB total; 56.5 B each GuardType 952.6 KB total; 97.7 B each HasType 910.4 KB total; 111.2 B each Send 705.0 KB total; 218.0 B each CCallWithFrame 689.9 KB total; 149.0 B each PushInlineFrame 559.8 KB total; 247.5 B each CCallVariadic 440.4 KB total; 32.0 B each EntryPoint 387.9 KB total; 17.8 B each CheckInterrupts 356.8 KB total; 66.7 B each GetIvar 326.3 KB total; 11.0 B each Jump 198.2 KB total; 8.0 B each CondBranch 167.6 KB total; 22.3 B each Return 163.5 KB total; 4.8 B each LoadField 149.0 KB total; 23.6 B each IsBitEqual 142.4 KB total; 100.0 B each HashAref 135.9 KB total; 16.3 B each Test 138705 PatchPoint 128879 PadPatchPoint 124942 Snapshot 45884 side-exit 40535 Const 34701 LoadField 30364 Jump 25234 CondBranch 22264 CheckInterrupts 17657 GuardType 16101 LoadArg 14081 EntryPoint 12892 RefineType 11106 SendDirect 9980 HasType 8541 Test 8385 Send 7693 Return 6938 GuardBitEquals 6699 LoadSelf plum% ``` or with just `--zjit-perf`: ``` plum% ruby ../tool/zjit_analyze_perf_map_size.rb /tmp/perf-22517.map 770.7 KB total; 192.7 KB each _app_views_stories__listdetail_html_erb__2872691580871714137_14696@/Users/emacs/Documents/code/yjit-bench/benchmarks/lobsters/app/views/stories/_listdetail.html.erb:1 452.0 KB total; 150.7 KB each _app_views_layouts_application_html_erb__3422508349557286544_14376@/Users/emacs/Documents/code/yjit-bench/benchmarks/lobsters/app/views/layouts/application.html.erb:1 232.9 KB total; 232.9 KB each _app_views_comments__comment_html_erb___3513273811879470227_16232@/Users/emacs/Documents/code/yjit-bench/benchmarks/lobsters/app/views/comments/_comment.html.erb:1 200.9 KB total; 50.2 KB each block-in initialize@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/erubi-1.13.1/lib/erubi.rb:135 194.0 KB total; 64.7 KB each parse@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/tzinfo-2.0.6/lib/tzinfo/data_sources/zoneinfo_reader.rb:345 191.8 KB total; 191.8 KB each _app_views_comments__comment_html_erb___3513273811879470227_15016@/Users/emacs/Documents/code/yjit-bench/benchmarks/lobsters/app/views/comments/_comment.html.erb:1 167.4 KB total; 167.4 KB each _app_views_comments__comment_html_erb___3513273811879470227_16488@/Users/emacs/Documents/code/yjit-bench/benchmarks/lobsters/app/views/comments/_comment.html.erb:1 165.9 KB total; 165.9 KB each _app_views_stories__listdetail_html_erb__2872691580871714137_15008@/Users/emacs/Documents/code/yjit-bench/benchmarks/lobsters/app/views/stories/_listdetail.html.erb:1 124.3 KB total; 31.1 KB each build_arel@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/relation/query_methods.rb:1751 120.9 KB total; 30.2 KB each _delegator_method@/Users/emacs/.rubies/ruby-zjit-rel-stats/lib/ruby/4.1.0+4/forwardable.rb:202 119.5 KB total; 29.9 KB each perform_query@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/connection_adapters/sqlite3/database_statements.rb:87 111.6 KB total; 55.8 KB each _app_views_stories_show_html_erb__2313005407901203172_15000@/Users/emacs/Documents/code/yjit-bench/benchmarks/lobsters/app/views/stories/show.html.erb:1 103.8 KB total; 25.9 KB each block-in generate@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activesupport-8.1.1/lib/active_support/delegation.rb:72 98.3 KB total; 98.3 KB each _app_views_users_show_html_erb___3973630128665404112_14592@/Users/emacs/Documents/code/yjit-bench/benchmarks/lobsters/app/views/users/show.html.erb:1 91.0 KB total; 45.5 KB each normalize_options@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/bundler-4.0.12/lib/bundler/dsl.rb:394 85.9 KB total; 21.5 KB each parse@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/tzinfo-2.0.6/lib/tzinfo/data_sources/posix_time_zone_parser.rb:37 74.5 KB total; 37.3 KB each generate@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activesupport-8.1.1/lib/active_support/delegation.rb:22 69.0 KB total; 17.3 KB each determine_template@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/actionview-8.1.1/lib/action_view/renderer/template_renderer.rb:17 67.5 KB total; 16.9 KB each block-in expand_from_hash@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/relation/predicate_builder.rb:88 65.2 KB total; 16.3 KB each block-in derive_offsets@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/tzinfo-2.0.6/lib/tzinfo/data_sources/zoneinfo_reader.rb:118 17 __callbacks@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activesupport-8.1.1/lib/active_support/callbacks.rb:70 12 _layout@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/actionview-8.1.1/lib/action_view/layouts.rb:329 5 __callbacks=@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activesupport-8.1.1/lib/active_support/callbacks.rb:70 5 default_url_options@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/actionpack-8.1.1/lib/action_dispatch/routing/url_for.rb:100 4 scope@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/associations/association.rb:108 4 default_render@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/actionpack-8.1.1/lib/action_controller/metal/implicit_render.rb:38 4 scope@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/associations/association_scope.rb:22 4 get_method_for_class@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/actionpack-8.1.1/lib/action_dispatch/routing/polymorphic_routes.rb:348 4 visit_Arel_Nodes_SqlLiteral@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/arel/visitors/to_sql.rb:755 4 block-in preprocess_order_args@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/relation/query_methods.rb:2105 4 strictly_lower?@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/bundler-4.0.12/lib/bundler/vendor/pub_grub/lib/pub_grub/version_range.rb:179 4 choose_compatible@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/bundler-4.0.12/lib/bundler/lazy_specification.rb:236 4 column_types@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/result.rb:168 4 local_search@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/bundler-4.0.12/lib/bundler/index.rb:65 4 method_defined_within?@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/attribute_methods.rb:187 4 join_scopes@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/reflection.rb:227 4 block-in expand_from_hash@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/relation/predicate_builder.rb:88 4 block-in convert_dot_notation_to_hash@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/relation/predicate_builder.rb:166 4 block-in references@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/activerecord-8.1.1/lib/active_record/relation/predicate_builder.rb:29 4 block-in materialize_dependencies@/Users/emacs/.gem/ruby/ruby-zjit-rel-stats/gems/bundler-4.0.12/lib/bundler/spec_set.rb:246 plum% ```
That attribute is C++23, not C23 ruby/prism@5e4f45712e
rb_vm_opt_getconstant_path() asserts on an inline constant cache hit that
the cached value equals a freshly recomputed constant chain:
VM_ASSERT(val == vm_get_ev_const_chain(ec, segments));
On a non-main ractor an IC hit is only allowed for shareable values
(IMEMO_CONST_CACHE_SHAREABLE). Reassigning a shareable constant is legal
from a non-main ractor, and it invalidates ICs asynchronously via
rb_clear_constant_cache_for_id() (ic->entry = NULL) without locking the
reader side. So the hit path can load a still-valid, previously-assigned
shareable value while another ractor concurrently reassigns the constant;
the subsequent recomputation then observes the newer value and the two
legitimately differ (a benign TOCTOU). Both values were validly assigned
to the constant at some point, so no correctness or memory-safety issue
results -- only this debug-only assertion trips.
Guard the equality check with rb_multi_ractor_p(), mirroring the existing
workaround in vm_cc_invalidate() (vm_callinfo.h). Reproducible with several
ractors reassigning the same frozen constant in a loop; NDEBUG builds are
unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pow5mult's b_cache publishes new cache nodes with ATOMIC_PTR_CAS (release), but both readers -- the fast-path check and the recheck before allocating -- were plain loads racing with that CAS: a data race in the C11 sense, and what TSan reports as `race:pow5mult` under parallel Ractor float<->string conversion. Harmless on current targets (the reader dereferences through the loaded pointer, so address dependency orders the accesses), but easy to make formally correct: read the cache head/next through RUBY_ATOMIC_PTR_LOAD. The lock-free CAS scheme is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Doing it in `rb_gc_obj_changed_slot_size` is wrong as the object could still be moved back because of `invalidate_moved_plane`.
Followup: #17631 Now that complex `RObject` always have an IMEMO/fields, we no longer need to have an external `st_table` to replicate the memory layout of `RObject`.
By handling complex shapes in `rb_shape_rebuild` we can merge the two complex cases (when `src` is complex and when `dest` becomes complex).
It used to be that multiple paths would need to clear the generic ivar reference, but it's no longer the case.
Share more code with `rb_copy_generic_ivar`.
Now share more code between the various cases: - Whether to copy all fields or just ivars - Whether to add or remove some capacity.
The layout is shared by T_DATA objects and overflowed RObjects, so name the Rust variant after SHAPE_ID_LAYOUT_EXTENDED rather than one of its users.
Directly update existing instance variables in the fields object used by overflowed RObjects. Keep shape-changing writes on the generic path so the owner and fields object retain their private layout and capacity bits.
Lightweight frames need C code to use CFP_ISEQ(...).
Match YJIT by checking interrupts only on backward jump, branchif, branchunless, and branchnil instructions. Forward branches cannot form loops and do not need polling.
gmp is enabled by default and setup/macos passes --with-gmp-dir to every job, so the plain check job already links -lgmp and the dedicated --with-gmp job duplicated it. Trims one of ten macOS jobs per event against the org-wide macOS concurrency cap of 50. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The dedicated cargo job occupied a runner from the org-wide macOS concurrency pool of 50 for less than a minute of work. Run the Rust unit tests as steps of the --enable-yjit entry instead, which stays shorter than the --enable-yjit=dev long pole. The steps are guarded with !cancelled() so a Ruby-side test failure does not mask the Rust result. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The full macOS fan-out (10 jobs, about 117 runner-minutes) repeats test tasks that macos.yml already runs from the git tree on every event; what is tarball-specific is only that the packaged tarball configures, builds and installs. Run a single check on macos-26 for pull requests and pushes, and keep the full matrix for workflow_dispatch, which is how the nightly schedule and release verification invoke it. Only macOS is trimmed: its org-wide concurrency pool of 50 runners saturates during peak hours and has been delaying release jobs, while Linux and Windows share a pool with headroom. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was by far the heaviest bundler spec file (~2500 example-seconds on Windows). Parallel spec runs distribute work per file, so a single huge file caps the achievable wall-clock time. Move its describe blocks verbatim into three files, keeping example count and structure unchanged. ruby/rubygems@a50eb931e5 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Its before(:context) shard warning fires on every run since the file was added without a shards.rb entry. ruby/rubygems@f9879f9015 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Recorded per-file runtimes show outdated_spec.rb, newgem_spec.rb and update_spec.rb each exceed the ideal per-worker load of a parallel run, and setup_spec.rb dominates standalone runs, so file-based distribution cannot go faster without splitting them. Blocks move verbatim and example sets are unchanged. newgem's shared examples move together with their consumers, with its top-level helpers duplicated into both files. ruby/rubygems@5d40766029 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
BSD/OS was discontinued in 2003 and the bsdi* target has been unbuildable for decades. This also removes the BROKEN_SETREUID and BROKEN_SETREGID fallbacks in process.c, which no other platform ever defined. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…afely Same relaxation as PR #17873: another ractor can invalidate a cc (klass = Qundef) while a call using it is in flight; cme_/call_ stay intact, so the call proceeds with the method resolved before the redefinition (same treatment as vm_cc_invalidate). Route the check through VM_CC_RACED_P(), which extensions see as a constant: ext/objspace/objspace_dump.c includes vm_callinfo.h and calls vm_cc_cme, and with assertions enabled at -O0 (--enable-yjit/zjit=dev on macOS) a direct rb_multi_ractor_p() there materializes references to the core-internal globals ruby_current_vm_ptr / ruby_single_main_ractor, which are hidden from extensions -- objspace.bundle then fails to load ("symbol not found in flat namespace '_ruby_current_vm_ptr'", the five JIT-dev CI failures on #17873). At -O1+ the compiler proves the first disjunct from the caller's Qundef guard and dead-codes the call, which is why only the -O0 dev jobs saw it. Verified both ways on the objspace_dump.c TU: PR state at -O0+RUBY_DEBUG has both unresolved globals; with VM_CC_RACED_P() it has none. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bumps the github-actions group with 1 update in the / directory: [ruby/setup-ruby](https://github.com/ruby/setup-ruby). Updates `ruby/setup-ruby` from 1.316.0 to 1.318.0 - [Release notes](https://github.com/ruby/setup-ruby/releases) - [Changelog](https://github.com/ruby/setup-ruby/blob/master/release.rb) - [Commits](ruby/setup-ruby@d45b1a4...8e41b36) --- updated-dependencies: - dependency-name: ruby/setup-ruby dependency-version: 1.318.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com>
Gem::Source#fetch_spec built the local spec cache path directly from the name tuple returned by a remote index, which is never validated for use as a path component. A crafted gem name containing path separators or `..` could therefore make fetch_spec write the downloaded gemspec bytes outside Gem.spec_cache_dir. Reject any spec name that is not a plain basename before constructing the cache path. ruby/rubygems@56ed326cb2 Co-Authored-By: Claude <noreply@anthropic.com>
Track explicit lengths while parsing command strings and comparing the current directory. This avoids relying on Ruby string terminators and makes separators in the generated argument buffer explicit.
Compare compiler attribute names using Ruby string lengths. Share attribute handling between the compiler paths to keep their behavior aligned.
Restore hidden operand strings and arrays with their dedicated resurrection functions so inspection preserves all contents.
Use the message length when looking for the receiver placeholder instead of requiring a NUL-terminated string.
Search only the gethostname buffer for its terminator so resizing the Ruby string does not read beyond the returned data.
Move the local fallback out of string.c so other source files can use memrchr on platforms that do not provide it.
Search and compare feature extensions within the Ruby string length so require does not rely on a terminator while classifying paths.
Create system errors from the filepath VALUE so the full path is retained without relying on a NUL terminator or a separate GC guard.
The Hash#[] and Array#product examples referenced undefined variables, the StringIO doc named a nonexistent File::RDRW constant, and Pathname#mkpath's call-seq advertised a positional permissions argument that the keyword-only implementation does not accept. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Set#include? built its receiver from an Array literal, so the `hash`/`eql?` semantics the surrounding text describes did not apply. Set#each wrote `sum => 6`, which is a rightward-assignment pattern match rather than an output comment. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add Proc#refined(mod, ...), which returns a new Proc that behaves like the receiver but with the refinements activated by the given modules in effect inside its body, without affecting the original Proc. This is an alternative to the previously proposed Proc#using (https://bugs.ruby-lang.org/issues/16461) that does not modify the existing block and needs no `using Proc::Refinements` declaration. Refinements are resolved at run time through the cref, so a refined proc gets two pieces of its own: * A cref carrying the additional refinements, built by activating the modules (rb_using_module_recursive, under the VM lock) on a duplicate of the block's captured cref. It is stored in a hidden ivar on the Proc and injected into the block frame on every proc-invocation path (Proc#call, yield from C methods, invokeblock/opt_call, and instance_eval/instance_exec/module_eval/class_eval via yield_under). The captured environment stays shared, so closure variables are still shared with the original Proc. `using` inside the body is rejected (CREF_REFINED_PROC), so the cref's refinements never change after publication. * A recursive deep copy of the block's instruction sequences (rb_iseq_dup_with_independent_caches) so the new Proc has its own inline method caches. This is required for correctness: the VM caches refinement method resolution per call site assuming a single lexical cref per iseq, so sharing the iseq would leak the refined methods into the original Proc. Coverage keeps measuring the copied iseq. The copy and the cref are memoized in a hidden identity Hash keyed by the source iseq (one slot per iseq, matched by the captured cref and the module arguments, accessed under the VM lock), so repeated calls at the same site reuse one copy and procs refined with the same modules share the iseq and the cref. A differing module set or a ruby2_keywords flag mismatch rebuilds the entry (with a performance warning). The refinements table of the memoized cref is frozen and marked shareable so the memo also serves other Ractors. Procs without an iseq block (Symbol#to_proc, C-function procs), procs created from methods, and procs that already have refinements are rejected with ArgumentError. The YJIT/ZJIT Rust bindings are regenerated for the new rb_proc_t field. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…YAMLSerializer Splitting the source on LF alone left a trailing CR on every line, so the empty-value check saw a non-empty scalar and folded nested mappings and sequences into a single plain scalar. YAML normalizes CRLF to LF during parsing, so the CR belongs to the line break, not the content. The existing CRLF test only covered a flat mapping, which happens to survive the stray CR. ruby/rubygems@067e0aa4bf Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ck scalar A literal block scalar cannot represent a CR because parsing normalizes CRLF line breaks to LF, so a string like a CRLF PEM certificate lost its CRs on roundtrip. Emit such strings as double-quoted scalars with escapes, matching Psych. This fixes TestGemPackage signing tests on Windows JRuby where to_pem returns CRLF line endings. ruby/rubygems@afc73d34d8 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )