asp/keep
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: Add rustdoc comments for text metadata plugin internals

Browse Source 
Co-authored-by: aider (openai/andrew/openrouter/sonoma-sky-alpha) <aider@aider.chat>
This commit is contained in:
Andrew Phillips
2025-09-10 13:57:51 -03:00
parent 099698e388
commit 9ad3c1d9c8
2 changed files with 123 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
PLAN.md
View File

@@ -71,20 +71,20 @@ Private helpers (e.g., internal `fn` without `pub`) are not flagged, as they don
- `register_filter_plugin()` and `get_available_filter_plugins()` functions: Partial.
- Overall: Global registry lacks docs.
8. **src/services/item_service.rs**
8. **src/services/item_service.rs** [DONE]
- `ItemService` struct: Partial.
- Many methods (`get_item`, `get_item_content`, `get_item_content_info`, `find_item`, `list_items`, `delete_item`, `save_item`, `save_item_from_mcp`, `get_compression_service`, `get_data_path`): Partial or no docs.
- `FilteringReader` struct and impl `Read`: No docs.
- Overall: Extensive but inconsistent docs.
9. **src/config.rs**
9. **src/config.rs** [DONE]
- Enums (`ColumnAlignment`, `ContentArrangement`, `TableStyle`, `TableColor`, `TableAttribute`): Partial or no docs.
- Structs (`TableConfig`, `ColumnConfig`, `ServerConfig`, `CompressionPluginConfig`, `MetaPluginConfig`, `Settings`): Partial.
- `new()` method on `Settings`: Partial.
- Helper methods (`get_server_password`, `server_password`, etc.): No docs.
- Overall: Config structs need better field-level docs.
10. **src/meta_plugin/text.rs**
10. **src/meta_plugin/text.rs** [DONE]
- `TextMetaPlugin` struct: No doc.
- `new()` function: No doc.
- Many helper functions (`count_text_stats`, `perform_binary_detection`, etc.): No docs.

131
src/meta_plugin/text.rs
View File

@@ -140,7 +140,13 @@ impl TextMetaPlugin {
}
/// Count words and lines in a text chunk, handling block boundaries correctly
/// Count words and lines in a text chunk, handling block boundaries correctly.
///
/// Processes UTF-8 data, tracks word transitions, and updates line length statistics.
///
/// # Arguments
///
/// * `data` - Byte slice of text content.
fn count_text_stats(&mut self, data: &[u8]) {
// Count lines (newlines) if needed
if self.track_line_count {
@@ -217,8 +223,17 @@ impl TextMetaPlugin {
}
}
/// Helper method to perform binary detection and return appropriate metadata
/// Returns (metadata, should_finalize) tuple
/// Helper method to perform binary detection and return appropriate metadata.
///
/// Uses the is_binary function to check the buffer and sets text-related outputs accordingly.
///
/// # Arguments
///
/// * `buffer` - Data to check for binary content.
///
/// # Returns
///
/// * `(Vec<MetaData>, bool)` - Metadata updates and whether content is binary.
fn perform_binary_detection(&mut self, buffer: &[u8]) -> (Vec<crate::meta_plugin::MetaData>, bool) {
let mut metadata = Vec::new();
let is_binary_result = is_binary(buffer);
@@ -259,14 +274,18 @@ impl TextMetaPlugin {
(metadata, is_binary_result)
}
/// Helper method to process the remaining UTF-8 buffer and finalize text statistics
/// Helper method to process the remaining UTF-8 buffer and finalize text statistics.
///
/// Calls count_text_stats with empty data to handle any pending UTF-8 bytes.
fn process_remaining_utf8_buffer(&mut self) {
if !self.utf8_buffer.is_empty() {
self.count_text_stats(&[]);
}
}
/// Helper method to handle the last line when tracking line lengths
/// Helper method to handle the last line when tracking line lengths.
///
/// Updates statistics for any unfinished line at EOF.
fn handle_last_line_for_length_tracking(&mut self) {
if self.track_line_lengths && self.current_line_length > 0 {
// Update max line length for the last line
@@ -285,7 +304,11 @@ impl TextMetaPlugin {
}
}
/// Helper method to output word count metadata
/// Helper method to output word count metadata.
///
/// # Returns
///
/// * `Option<MetaData>` - Metadata entry if tracking is enabled.
fn output_word_count_metadata(&self) -> Option<crate::meta_plugin::MetaData> {
if self.track_word_count {
crate::meta_plugin::process_metadata_outputs(
@@ -298,7 +321,11 @@ impl TextMetaPlugin {
}
}
/// Helper method to output line count metadata
/// Helper method to output line count metadata.
///
/// # Returns
///
/// * `Option<MetaData>` - Metadata entry if tracking is enabled.
fn output_line_count_metadata(&self) -> Option<crate::meta_plugin::MetaData> {
if self.track_line_count {
crate::meta_plugin::process_metadata_outputs(
@@ -311,7 +338,11 @@ impl TextMetaPlugin {
}
}
/// Helper method to output max line length metadata
/// Helper method to output max line length metadata.
///
/// # Returns
///
/// * `Option<MetaData>` - Metadata entry if enabled and data exists.
fn output_max_line_length_metadata(&self) -> Option<crate::meta_plugin::MetaData> {
if self.output_line_max_len && self.line_count_for_stats > 0 {
crate::meta_plugin::process_metadata_outputs(
@@ -324,7 +355,13 @@ impl TextMetaPlugin {
}
}
/// Helper method to output mean line length metadata
/// Helper method to output mean line length metadata.
///
/// Computes average line length and rounds to nearest integer.
///
/// # Returns
///
/// * `Option<MetaData>` - Metadata entry if enabled and data exists.
fn output_mean_line_length_metadata(&self) -> Option<crate::meta_plugin::MetaData> {
if self.output_line_mean_len && self.line_count_for_stats > 0 {
let mean_len = self.total_line_length as f64 / self.line_count_for_stats as f64;
@@ -340,7 +377,13 @@ impl TextMetaPlugin {
}
}
/// Helper method to output median line length metadata
/// Helper method to output median line length metadata.
///
/// Sorts line lengths and computes median (average of middle two for even count).
///
/// # Returns
///
/// * `Option<MetaData>` - Metadata entry if enabled and data exists.
fn output_median_line_length_metadata(&self) -> Option<crate::meta_plugin::MetaData> {
if self.output_line_median_len {
if let Some(lengths) = &self.line_lengths {
@@ -365,7 +408,13 @@ impl TextMetaPlugin {
}
/// Helper method to output word and line counts
/// Helper method to output word and line counts.
///
/// Finalizes pending data and collects all enabled text statistics metadata.
///
/// # Returns
///
/// * `Vec<MetaData>` - List of metadata entries.
fn output_word_line_counts(&mut self) -> Vec<crate::meta_plugin::MetaData> {
// Process any remaining data in utf8_buffer
self.process_remaining_utf8_buffer();
@@ -408,16 +457,38 @@ impl TextMetaPlugin {
}
impl MetaPlugin for TextMetaPlugin {
/// Checks if the plugin has been finalized.
///
/// # Returns
///
/// `true` if finalized, `false` otherwise.
fn is_finalized(&self) -> bool {
self.is_finalized
}
/// Sets the finalized state of the plugin.
///
/// # Arguments
///
/// * `finalized` - The new finalized state.
fn set_finalized(&mut self, finalized: bool) {
self.is_finalized = finalized;
}
/// Updates the plugin with new data chunk.
///
/// Accumulates data for binary detection (if pending) or text statistics.
/// Finalizes early if binary content is detected.
///
/// # Arguments
///
/// * `data` - Byte slice of content chunk.
///
/// # Returns
///
/// * `MetaPluginResponse` - Current metadata and finalized status.
fn update(&mut self, data: &[u8]) -> MetaPluginResponse {
// If already finalized, don't process more data
if self.is_finalized {
@@ -500,6 +571,14 @@ impl MetaPlugin for TextMetaPlugin {
}
}
/// Finalizes the plugin and emits all pending text statistics.
///
/// Performs binary detection if not done, then outputs enabled statistics.
/// Handles head/tail options for content preview (future implementation).
///
/// # Returns
///
/// * `MetaPluginResponse` - Final metadata and finalized status.
fn finalize(&mut self) -> MetaPluginResponse {
// If already finalized, don't process again
if self.is_finalized {
@@ -607,18 +686,38 @@ impl MetaPlugin for TextMetaPlugin {
}
}
/// Returns the type of this meta plugin.
///
/// # Returns
///
/// `MetaPluginType::Text`.
fn meta_type(&self) -> MetaPluginType {
MetaPluginType::Text
}
/// Returns a reference to the outputs mapping.
///
/// # Returns
///
/// A reference to the `HashMap` of outputs.
fn outputs(&self) -> &std::collections::HashMap<String, serde_yaml::Value> {
self.base.outputs()
}
/// Returns a mutable reference to the outputs mapping.
///
/// # Returns
///
/// A mutable reference to the `HashMap` of outputs.
fn outputs_mut(&mut self) -> &mut std::collections::HashMap<String, serde_yaml::Value> {
self.base.outputs_mut()
}
/// Returns the default output names for this plugin.
///
/// # Returns
///
/// Vector of default output field names.
fn default_outputs(&self) -> Vec<String> {
vec![
"text".to_string(),
@@ -630,10 +729,20 @@ impl MetaPlugin for TextMetaPlugin {
]
}
/// Returns a reference to the options mapping.
///
/// # Returns
///
/// A reference to the `HashMap` of options.
fn options(&self) -> &std::collections::HashMap<String, serde_yaml::Value> {
self.base.options()
}
/// Returns a mutable reference to the options mapping.
///
/// # Returns
///
/// A mutable reference to the `HashMap` of options.
fn options_mut(&mut self) -> &mut std::collections::HashMap<String, serde_yaml::Value> {
self.base.options_mut()
}