First Experiments in Prompt-Based RAG Construction Using Conversational History

1. Introduction

This report summarises initial work on transforming AI–human conversational history and contemplative study transcripts into structured, searchable segments suitable for retrieval via a Retrieval-Augmented Generation (RAG) system. The aim is not simply to enable question answering or summarisation, but to investigate whether past exchanges—when segmented and described via prompt-guided interpretation—can be reintegrated into a dynamic field of meaning.

This effort is situated within a broader project to model mental states and appearances using Mahāmudrā-informed prompt design and meaning-space alignment, drawing on contemplative traditions (Wallace, 2006; Varela et al., 1991) and interpretability-focused prompt engineering (Reynolds & McDonell, 2021).

2. Source Material

The primary sources for this experiment included a set of ChatGPT conversation exports and transcriptions from Mahāmudrā study sessions, particularly from The Ocean of Definitive Meaning. These included teaching presentations, guided meditations, and group dialogues led by senior practitioners. The material was heterogeneous and multi-thematic, spanning practice reflections, philosophical inquiry, and experiential guidance.

Exports and transcripts were converted to plaintext and lightly cleaned to remove system artefacts. No structural markup (e.g., speaker tagging) was used beyond file-level naming and basic HTML-to-text conversion.

3. Segmentation and Classification

Each file was processed using a pipeline that:

  1. Sampled three positions in the conversation or transcript (beginning, middle, end)
  2. Submitted these to a local Mistral model with a classification prompt (prompt_classify.txt)
  3. Assigned soft, non-exclusive labels such as “study presentation,” “guided meditation,” “reflective commentary,” etc.

Following classification, the file was segmented using a custom segmentation script that:

  • Respected paragraph and sentence structure
  • Used a contemplative segmentation prompt to detect shifts in mental mode or appearance (cf. Tsur & Rappoport, 2019)
  • Preserved full original text in each segment for traceability

Output: A folder of segment-level JSON files under segment_cut/, each with:

  • Unique ID
  • Full text
  • Source filename and classification metadata

4. Descriptor Generation

Each segment was passed through a two-pass prompt-based analysis pipeline using Mistral:

First Pass (generator.py with prompt0.txt)

  • Prompted to identify: mental_state, appearance, transition_trigger, and additional symbolic or energetic dimensions
  • Included a comment field and confidence scores for interpretive elements
  • Flagged segments for second pass if output was incomplete or ambiguous ("gpt35": true)
  • Output stored in rag_cut/

Second Pass (augmenter.py with prompt1.txt)

  • Used enhanced prompt with first-pass output as context
  • Refined descriptors, filled missing fields, improved semantic subtlety
  • Allowed affirmation, light disagreement, or elaboration of the original interpretation
  • Ensured schema compliance for downstream ingestion (cf. Zhou et al., 2023)

5. Embedding and Query System

Final validated segments were embedded using embedder.py into a ChromaDB vector store under /mnt/1TBSTORE/RAG_VECTORS/. A lightweight Flask-based web server allows semantic querying:

  • User submits a reflective prompt (e.g., “describe a moment of epistemic tension”)
  • Vector query retrieves top-matching segments
  • UI displays original text, JSON descriptors, and source filename

This interface functions more like a reflective browser than a traditional search tool. The meaning-field rather than surface keyword match is prioritised (cf. Lewis et al., 2020).

6. Observations

  • Prompt fragility: Small changes in prompt wording (e.g., “mental tone” vs “appearance”) significantly altered output structure.
  • Surprising coherence: The LLM often identified subtle shifts that aligned with the user’s recollected state—even from months-old conversations.
  • Failure modes: Occasionally, the model reified metaphors or hallucinated transitions. These were mostly caught by the schema validator.
  • Interpretive power: When querying the resulting vector store, surprising cross-file connections emerged—suggesting that the structure of inquiry over time can be mapped.

6.1 Live Query Demonstration: Cross-Model Retrieval

Query submitted to the RAG vector interface:
“moments of tension or contraction in the appearance of mind when insight is beginning to arise”

Two responses were retrieved via semantic similarity, one from a local Mistral model and the other from GPT-3.5 (OpenAI). Both were aligned with Mahāmudrā semantics and showed strong internal consistency:

🔹 Mistral (local) response:

“Observe the tension with an open and compassionate heart… see it not as obstacle but as doorway… remain with the awareness that is untouched… the luminous nature of mind.”

🔹 GPT-3.5 (OpenAI) response:

“Tension often signifies a point of transition… observe with gentle awareness… uncover the empty and luminous nature… insight unfolds naturally in the spaciousness of mind.”

Both responses reflect transitions marked in the JSON descriptors by fields such as:

  • transition_vector: [“contraction → openness”]
  • mental_state: [“tension”, “clarity”, “curiosity”]
  • symbolic_field: [“threshold”, “luminous space”]
  • comment: capturing contemplative tone and structure

This demonstrates that the segment-based RAG system, guided by meaning-space prompts and Mahāmudrā-informed schema, yields reliable and resonant retrievals across architectures.

7. Conclusion and Next Steps

This first round of experiments confirms that meaning-preserving segmentation and Mahāmudrā-informed prompt descriptors can turn informal conversational and structured contemplative data into a coherent semantic field for exploration.

Next steps include:

  • Extending the system to canonical texts and more diverse transcript types
  • Comparing different prompt strategies across identical segments
  • Enabling bi-directional feedback from query results back into prompt tuning
  • Incorporating Claude exports and broader multi-modal sources (e.g., audio transcriptions), allowing for comparison of stylistic and epistemic differences across AI interlocutors and between AI and human-led material

These results support the viability of a contemplatively aligned RAG system as both a retrieval tool and a mirror for structured awareness.

References

  • Lewis, P., et al. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. NeurIPS.
  • Reynolds, L., & McDonell, K. (2021). Prompt programming for large language models: Beyond the few-shot paradigm. arXiv:2102.07350.
  • Tsur, O., & Rappoport, A. (2019). What’s in a headline? Hierarchical attention networks for joint document classification and summarization. NAACL.
  • Varela, F. J., Thompson, E., & Rosch, E. (1991). The Embodied Mind: Cognitive Science and Human Experience. MIT Press.
  • Wallace, B. A. (2006). The Attention Revolution: Unlocking the Power of the Focused Mind. Wisdom Publications.
  • Zhou, D., et al. (2023). Least-to-most prompting enables complex reasoning in large language models. ICLR.