Progress (PFS)
Progress Federal Solutions (PFS) Products.

MarkLogic (ML). -- (The Secure database where the data is stored -- The Hanger)

• BLUF: A Multi-Model Database. It’s a library that can store a logbook, a voice recording, a spreadsheet, or a photo without needing to rewrite them first. It keeps the original data but puts it all under one roof. -- By leveraging a multi-model integration, this eliminates ETL (Extract, Transform, Load) friction.

• Data Pipeline: -- BLUF: Using MarkLogic (emulating the Databricks + Azure Data Fabric (ADF) stack)

  1. -- \/ -- [ External Systems ]  

  2. -- \/ -- (Batch / Continuous Ingest)

  3. -- \/ -- MARKLOGIC (Single Unified Platform) -- Inside the MarkLogic Platform.

     1. Ingestion -- MLCP, DMSDK, REST

        • (Replaces ADF data movement)

     2. Pipeline & Harmonization -- Data Hub Framework (DHF)

        • (Raw / Staging ──> Harmonized / Final)

        • (Replaces Delta Medallion Bronze ──> Silver) ~ Note:  This is the Medallion Architecture (Bronze:  Raw data ──> Silver: Cleaned/Conformed ──> Gold: Curated/Aggregated). It is an architectural design pattern (or data organization framework), not a distinct software product. It was popularized and heavily evangelized by Databricks as the recommended way to structure a Modern Data Lakehouse.   

     3. Analytical Layer -- TDE + Optic API / SQL Views

        • (Indexes document nodes into virtual row stores)

        • (Replaces Spark SQL / Gold Marts) ~ Note: Databricks heavily relies on both.

          1. Spark SQL -- A core module of Apache Spark, which is an open-source, distributed general-purpose cluster-computing framework originally developed at UC Berkeley's.

          2. Gold Mart -- (or Data Mart) is an industry-standard architectural concept, not a platform, proprietary tool, or branded product.

     4. (4) Operational Serving & Security 

        • (ACID, Sub-second search, Element-Level Security) 

• Features: (7)

  1. Semantics:

     1. Semantic Data Layer: Built-in semantics — RDF triple store, SPARQL, inference. This is the technical basis for the OMS paper's "ontology workbench."

  2. Advanced Security

  3. Multi-Model Domain Data Integration:

     1. Multi-model -- Unification -- documents, graph (RDF triples), geospatial data, and full-text search living in one engine, queryable together. Before MarkLogic, you'd typically bolt four different systems together.

  4. Cross-Classification:

  5. Data Governance:

  6. Secure Data Consolidation:

  7. ACID -- (Trust Layer) -- It what makes the multi-model great for governments and banks. "Store everything together." ACID is the trust layer that lets the real headline — one platform, all data shapes, all searchable, NoSQL. ACID transactions, multi-models (document, graph, semantic, geospatial), RDF triple store with semantic enrichment, ontology workbench.

     1. Atomicity: a transaction either fully completes or fully fails — never half-written.

     2. Consistency: the database moves from one valid state to another; rules are never violated mid-write.

     3. Isolation: concurrent transactions don't see each other's partial work.

     4. Durability: once committed, data survives crashes and power loss.

  8. Native enterprise search — everything indexed and searchable out of the box; no separate search engine to bolt on and keep in sync.

  9. Granular security — element-level access control and redaction. This is what makes the ABAC story implementable: permissions down to individual elements, not just whole documents.

  10. Geospatial — native geo indexes and queries, same engine.

  11. Scale and resilience — clustering, replication, failover. The DDIL story stands on this.

• Interoperable w/: Semaphore;

• Value and Impact:

  1. The Core Limitation: MarkLogic has a known limitation where it has never natively supported cross-platform backups or cross-architecture replication between different hardware platforms (such as x86 and ARM/Graviton).

  2. Engineering Constraint: Because Engineering is unlikely to address this natively, it creates a massive architectural blocker for moving production systems to Graviton within the strict 3 to 4-hour downtime limit.

MarkLogic Server (MLS).

• Admin Login:

o  Admin: AdminJSG

o  Admin Pwr: MAR1! . . .

o  Realm: Public (is default)

o  Wallet Pwr: MAR1! . . .

o  Encrypt Security Database & Encrypt Audit Log: All uncheck.

• MLS URLs:

o  Local Host: 8001 (Server Install) -- http://localhost:8001/initialize-admin.xqy -- [Done]

o  Local Host: 8001 (Admin) -- http://localhost:8001/default.xqy?

o  Download MLS: https://www.marklogic.com/product/getting-started 

o  ML Documents (Content Hub): https://docs.progress.com/category/marklogic-content-hub 

Agentic RAG (PAR): -- (The "Concierge Agent" that delivers the output. PAR needs Semaphore!)

• BLUF: A SaaS-level app like Google's NotebookLM. See Google NotebookLM vs Progress Agentic RAG (PAR). -- See "Semaphore."

• Analogy: You ask, "Can we sue for a breach of contract based on this 2012 case?" The Assistant (PAR) uses Semaphore (the map) to know what shelves, to pull different files, reads through them, thinks about how they relate to the current problem, and writes a summary (output) to the user. 

• AV-2:

  1. Citations: Has citations telling you where an answer came from (may not be from an AuthS). Enterprises also need to know how reliable the answer is and how to improve it over time. When accuracy is business-critical, most teams want measurable quality, not just links. -- Best: To combine "Citations" and "AuthS." 

• Cost: $700/mo // Enterprise level can be 6-figures.

Chef.

• BLUF: Is about Configuration Management / Policy as Code (ensuring the machine is built correctly up to standard requirements). 

• Analogy: Chef acts like a robotic janitor and technician. Before every experiment, it automatically sets the room to the exact same temperature, scrubs the floors, and calibrates the scales to 0.000g. This is "Policy as Code." It ensures that every AI test happens in a perfectly "clean room" so the results are actually valid. 

• Interoperable w/: Flowmon;

• Key Features & Analysis:

  1. Action Chaining (Whatever): Enables complex workflows where scripts, OS commands, or Cookbooks are run sequentially based on conditions (e.g., only installing software if an API confirms approval).

  2. Scheduling Jobs (Whenever): Uses the RFC 5545 specification to schedule tasks like Java updates on specific days or at set intervals, with the ability to define exceptions.

  3. Controlled Rollout (Wherever): Automates deployments (like Nginx) across server groups to minimize downtime, allowing for instant rollbacks if failures occur.

  4. Security Orchestration: Integrates with Chef InSpec for automated, continuous compliance scanning (e.g., STIG profiles) and rapid incident response, such as automatically shutting down compromised nodes.

Flux.

• What it is: A data migration/movement utility used to transition data between systems.

• Value and Impact:

  • The Bottleneck: Using box stock/standard Flux for a full data migration from an x86 cluster to a Graviton cluster is far too slow, taking days or even weeks.

  • Alternative Solutions / Confusion: There is internal debate regarding its application. James Clippinger initially proposed using "Flux" like a classic master-plus-incremental backup approach to transition the database during an outage window, while Abby Potter / Alem noted that TicTac’s custom implementation is too complex to leverage. Clippinger later clarified that "box stock Flux" (run in parallel with customizations) could handle the use case, though the team initially confused it with flex rep.

Flex Rep (Flexible Replication).

• What it is: A replication feature/tool within the ecosystem (confused momentarily with Flux in the email thread).

• Value and Impact:

  • Clarified as the feature/term Abby Potter actually meant when discussing Alem's feedback on TicTac's implementation complexity.

Flowmon.

• BLUF: (Network Observability) It Is about Network Telemetry (watching how data moves to ensure it's safe). Catching "incorrect assumptions" (anomalies) in real-time. -- Uses AI/ML to monitor network performance and detect hidden security threats. 

• Analogy: (The Security Camera & Sensor): Flowmon is like a high-tech sensor system in the vents and a 4K security camera. It watches the "flow" of everything in the lab. If the AI starts behaving strangely—like trying to send data where it shouldn't (a gas leak)—Flowmon detects the anomaly immediately and trips the alarm before it reaches the outside world.

• Interoperable w/: Chef;

• Key Features & Analysis:

  • Anomaly Detection Service (ADS): An AI-powered engine that identifies "insider threats" and zero-day attacks by detecting subtle deviations in network traffic behavior that traditional perimeter tools often miss.

  • Network Performance Monitoring and Diagnostics (NPMD): Provides deep visibility into hybrid environments to measure application versus network response times, helping to identify bottlenecks like jitter or packet loss.

  • Integration with WhatsUp Gold: Consolidates network telemetry and infrastructure monitoring into a single interface, reducing the "mean time to repair" (MTTR) by allowing admins to see traffic analysis directly alongside device health.

GovSupport -- (Not a PFS Tool).

• What it is: A support ticketing system or portal used to log formal support requests.

• Value and Impact:

  • Formal Documentation & SLAs: Cori Moore directs Troy Gilbert to have Alem open a formal ticket using Govsupport specifically for "TicTac."

  • This ensures that the performance metrics gathered by the team are officially documented, backed by specific Service Level Agreements (SLAs), and tracked through proper support channels.

Kemp Loadmaster.

• BLUF: An Advanced Delivery Controller (ADC) used for optimizing application availability and security. 

• Analogy:

• Interoperable w/:

• Key Features & Analysis:

  1. Global Server Load Balancing (GSLB): Distributes traffic across geographically dispersed data centers to ensure users connect to the closest server, reducing latency and providing disaster recovery.

  2. Zero Trust Access Gateways (ZTAG): Implements a "never trust, always verify" model for applications. It uses identity provider (iDP) integration and multi-factor authentication (MFA) to secure access based on user location and group membership.

  3. Content Switching: Allows the LoadMaster to route traffic based on the specific content of a request (Layer 7), such as sending URL requests for images to one server and database queries to another.

  4. Health Checks & Algorithms: Continually monitors "Real Servers" to ensure they are online; if a server fails, the LoadMaster uses algorithms like Round Robin or Least Connections to reroute traffic instantly.

Knowledge Graphs (aka "GraphRAG") -- (Is a "Process")

• BLUF: A RAG system that focuses on relationships and logical flow of the data. Identifies dependencies and critical paths. It doesn't just store the data; it stores the relationships between the data.  -- VALUE: Because it uses Nodes (entities) and Edges (relationships), it explicitly models dependencies like "Task A depends on Resource B" or "Service X is a child of Server Y". 

• Tools:

  1. MarkLogic: This is a multi-model database that serves as the storage and execution layer for the graph. It natively stores "RDF triples" (the standard format for graph data)—and allows you to search and query those relationships in real time.

  2. Semaphore: This is a Semantic AI platform that acts as the "brain" for the knowledge graph. It allows users to create, manage, and enrich Knowledge Models (such as taxonomies and ontologies) that define the relationships between different pieces of information. 

• Analogy: 

  1. Dependency -- (1) What happens if we shut down Server A?", a Knowledge Graph can trace the path to show every application and user that will lose access (that will be effected by the shut down). (2) Testing -- a failed AI test across the entire mission “Kill Chain,” rather than just seeing the failure as an isolated event. 

  2. Critical Path -- Can calculate the Critical Path by analyzing the sequence and constraints of those tasks, showing you exactly which step, if delayed, will push back the entire mission. 

MarkLogic (ML). -- (The Secure database where the data is stored -- The Hanger)

• BLUF: A Multi-Model Database. It’s a library that can store a logbook, a voice recording, a spreadsheet, or a photo without needing to rewrite them first. It keeps the original data but puts it all under one roof. -- By leveraging a multi-model integration, this eliminates ETL (Extract, Transform, Load) friction.

• Interoperable w/: Semaphore;

Semaphore. -- 

• BLUF: An "enterprise-level" RAG system validation that doesn't just find "text"; it understands the relationships and meaning behind the data. -- The Manifest/Map to the data (Source of Truth).

  1. Assures "The Source of Truth" w/no "Hallucinations"

  2. The Map / Manifest to the data / The GPS who knows every road, shortcut, & speed limit.

  3. PAR is the "Driver" who needs the GPS to get to their destination.

  4. Semaphore knows that "A" and "B" are related, but can't look at the two concepts and say, "Because these are related, we should probably warn this department." PAR (the Smart Agent) is the one that does the "thinking" and "reasoning" and "delivering" of the output

• Analogy: (1) -- Acts as a master translator that sits above the library. It realizes that when the French team says "Freins" and the English team says "Brakes," they are talking about the exact same safety component. It understands the concept of brakes across the entire airport. It "Retrieves" the right "Augmented" info so the "Generation" (the answer) is 100% accurate and consistent across the whole enterprise. (2) -- Acting like a librarian. It knows (mapped) where to find the book you need. The librarian has a "Map of Knowledge" (a taxonomy) to your needs in the library. 

• Architecture Page (P): https://marklogic365.sharepoint.com/sites/SL-SolutionArchitecture2   

• Semantic Layer: Makes data understandable, standard, structured.

• Interoperable w/: MarkLogic;

• Value -- Semaphore (as the metadata) uses MarkLogic (ML) as the SoT and PAR as the concierge.  ML is the KB w/out moving the data from ML. Semaphore assures the "Source of Truth (SoT)" w/out "Hallucinations" to PAR. Semaphore does this by generating metadata (as RDF Triples) that describes the data in ML. Instead of moving the data from ML to a new location, Semaphore stores these "Triples" directly back into ML. ML is a multi-model database, meaning it can store your original documents (JSON/XML) and the new semantic relationships (Graph) in the same database. -- PAR is the "Concierge Agent" that looks at Semaphore (Manifest/Map/SoT) to find exactly where the content is located in ML (database/Hangar). Then PAR delivers the output to the client.