feat: plugin-declared parallel execution, switch to env_logger, update deps
Parallel execution (opt-in via MetaPlugin::parallel_safe): - Add Send bound to MetaPlugin, parallel_safe() method (default false) - Override to true in digest, tokens, exec, magic_file plugins - MetaService: std::thread::scope for initialize_plugins and process_chunk - Extract plugins via NullMetaPlugin sentinel + std::mem::replace (no unsafe) - Panic tracking: join errors logged, NullMetaPlugin restored and finalized - MetaPluginExec: Box<dyn Write> -> Box<dyn Write + Send> - SendCookie wrapper for libmagic Cookie with unsafe impl Send Logging (stderrlog -> env_logger): - Custom format: [SSSSSS.mmm] LEVEL [module:] message (time-since-start ms) - Default level: Warn (matches previous behavior) - -v: Debug, -vv+: Trace, -q: off - -vv+ shows module path Maintenance: - Bump deps: thiserror 2.0, config 0.15, dns-lookup 3.0, lz4_flex 0.12, ringbuf 0.4, rand 0.9, lazy_static 1.5, env_logger 0.11 - Update Cargo.lock (186 transitive packages) - Clippy fixes: is_multiple_of, to_string_in_format_args, collapsible_if - Fix double-counting bug in TokensMetaPlugin::update - Fix schema description using plugin.description() Co-Authored-By: opencode <noreply@opencode.ai>
This commit is contained in:
@@ -318,7 +318,7 @@ pub fn process_metadata_outputs(
|
||||
})
|
||||
}
|
||||
|
||||
pub trait MetaPlugin
|
||||
pub trait MetaPlugin: Send
|
||||
where
|
||||
Self: 'static,
|
||||
{
|
||||
@@ -488,6 +488,17 @@ where
|
||||
""
|
||||
}
|
||||
|
||||
/// Returns true if this plugin can execute concurrently with other
|
||||
/// parallel-safe plugins.
|
||||
///
|
||||
/// Plugins that do significant per-chunk work (hashing, tokenization,
|
||||
/// piping to child processes) should return true. The MetaService will
|
||||
/// run all parallel-safe plugins in separate threads per phase, then
|
||||
/// process results sequentially.
|
||||
fn parallel_safe(&self) -> bool {
|
||||
false
|
||||
}
|
||||
|
||||
/// Builds the schema for this plugin from its options and outputs.
|
||||
///
|
||||
/// Default implementation infers option types from YAML values and
|
||||
|
||||
Reference in New Issue
Block a user