Moving from elasticsearch single node to cluster

[lcia-projects]

i'm pretty familiar with elasticsearch 8.. less familiar with docker. I need to build a es cluster with a t-pot hive install.
i'd like to have the tpot hive install on a fast ssd drive, but also have a couple of es nodes with larger slower spinning drives for long term storage of logs.

so.. in theory i'd like it to look something like this:

---- Main Tpot22-Hive Install on ssd drive(node 1, master data node) (hot node)
---- Elasticsearch 8 data node on ssd drive (master capable,data node 2) (hot node)
---- Elasticsearch 8 data node on large spinning disk (data node 3) (warm node)
---- Elasticsearch 8 data node on large spinning disk (data node 4) (warm node)

ILM policy to move data from hot nodes to warm nodes after 60 days.

my questions:

• is the /opt/tpot/docker/elk/docker-compose.yml and the /opt/tpot/docker/elk/elasticsearch/dist what i need to modify?
• with docker and how it does internal networking between contains and the outide world.. what do i need to open/or do to allow other virtual machines to communicate with ports 9200 and 9300 of the elasticsearch docker container? right now i think only 127.0.0.1 can do that.

[Ganesh-403]

@lcia-projects
What you're proposing is certainly possible from an Elasticsearch perspective, but it goes beyond the deployment model that T-Pot is designed and documented for.

1. Can T-Pot Hive be converted from a single-node Elasticsearch deployment to a cluster?

Yes, but it would require manually customizing the Elasticsearch deployment.

T-Pot ships with a single Elasticsearch node as part of its Docker Compose stack. The officially documented scaling model is Hive + Sensor, where additional hosts run honeypots and forward logs to the Hive—not multiple Elasticsearch data nodes. I couldn't find documentation describing a supported multi-node Elasticsearch cluster within T-Pot itself. :contentReference[oaicite:0]{index=0}

So, from what I can tell, you'd be maintaining a custom Elasticsearch deployment alongside T-Pot.

---

2. Are docker-compose.yml and the Elasticsearch configuration the right places to modify?

Yes.

The Elasticsearch service definition in the Docker Compose stack and the Elasticsearch configuration (elasticsearch.yml or its equivalent environment variables) are where you'd configure:

• cluster.name
• node.name
• node.roles
• discovery.seed_hosts
• cluster.initial_master_nodes

These are the standard settings required for multi-node cluster formation. :contentReference[oaicite:1]{index=1}

---

3. How do the other nodes communicate?

By default, T-Pot binds Elasticsearch to localhost (e.g. 127.0.0.1:64298 -> 9200) to reduce the attack surface.

For a cluster, the nodes must be able to communicate over the transport interface (9300) and the HTTP interface (9200, if clients or Kibana need it).

That means you'll need to:

• bind Elasticsearch to a routable interface instead of 127.0.0.1,
• configure network.host,
• configure discovery.seed_hosts,
• allow TCP 9300 between cluster members,
• allow TCP 9200 where appropriate (typically only for clients/Kibana). :contentReference[oaicite:2]{index=2}

Be aware that once you set network.host to a non-loopback address, Elasticsearch switches into production mode and enforces additional bootstrap checks. :contentReference[oaicite:3]{index=3}

---

4. Hot/Warm architecture

Your proposed layout:

• SSD hot nodes
• HDD warm nodes
• ILM after 60 days

is a standard Elasticsearch architecture and is supported through Index Lifecycle Management (ILM) and node roles/data tiers. That's an Elasticsearch feature rather than a T-Pot feature. :contentReference[oaicite:4]{index=4}

---

Recommendation

If your primary goal is long-term retention, I'd consider keeping the T-Pot Hive as-is and forwarding or replicating the indices to a separate Elasticsearch cluster dedicated to storage and analytics. This keeps your T-Pot installation much closer to the supported configuration while allowing you to scale Elasticsearch independently.

If you do decide to cluster the embedded Elasticsearch, expect to maintain those Docker Compose and Elasticsearch configuration changes yourself, as future T-Pot updates may overwrite or require reapplying them.

If this solves your problem, feel free to mark it as the accepted answer so others can find it easily.

[dmille6]

thank you for your response. what I've ended up doing is keeping the TPOT vm pretty stock and unmodified so its easy to update.

I created a completely new and separate ES cluster, and I modified the log stash http_input.conf to ship the data to both. I created 2 pipelines so if one fails it doesnt shut down the whole system. it's a little fault tolerant.

I keep data on the HIVE for 60 days, and the cluster holds it much longer for trend analysis.

its been chugging along for a couple of years now.

I have created some connectors for OpenCTI that pull data from TPOT into OpenCTI then share it out as a STIX/TAXII feed to partners.

it works well.

[Ganesh-403]

What you're proposing is definitely possible from an Elasticsearch perspective, but it goes beyond the deployment model that T-Pot is designed and documented for.

1. Can T-Pot Hive be converted from a single-node Elasticsearch deployment to a cluster?

Yes, but it requires manually maintaining a custom Elasticsearch deployment.

T-Pot's documented scaling model is Hive + Sensor, where additional sensor nodes forward their data to a central Hive. I couldn't find documentation describing a supported multi-node Elasticsearch cluster embedded within a T-Pot deployment. :contentReference[oaicite:0]{index=0}

So from what I can tell, if you cluster the bundled Elasticsearch yourself, you'll be responsible for maintaining those changes across future T-Pot updates.

2. Are docker-compose.yml and the Elasticsearch configuration the right places to modify?

Yes.

You would need to modify both the Docker Compose configuration and the Elasticsearch node configuration (elasticsearch.yml or the equivalent environment variables) to configure standard cluster settings such as:

cluster.name:
node.name:
node.roles:
discovery.seed_hosts:
cluster.initial_master_nodes:

These are the standard Elasticsearch settings used for cluster formation. :contentReference[oaicite:1]{index=1}

3. How do the nodes communicate?

By default, T-Pot exposes Elasticsearch very conservatively to reduce the attack surface.

For a multi-node cluster, the nodes must be able to communicate over the Elasticsearch transport layer (TCP 9300), and clients such as Kibana or Logstash will typically communicate over HTTP (TCP 9200).

That means you'll generally need to:

• bind Elasticsearch to a routable interface instead of localhost
• configure network.host
• configure discovery.seed_hosts
• allow TCP 9300 between all cluster members
• expose TCP 9200 only to trusted clients as needed

Keep in mind that once network.host is set to a non-loopback address, Elasticsearch enters production mode and enforces additional bootstrap checks. :contentReference[oaicite:2]{index=2}

4. Hot/Warm architecture

Your proposed architecture:

• Hive + SSD (hot)
• Additional SSD node (hot)
• HDD nodes (warm)
• ILM moving indices after 60 days

is a standard Elasticsearch deployment pattern.

Hot/warm data tiers and ILM are native Elasticsearch features and are well suited for log retention workloads. :contentReference[oaicite:3]{index=3}

Recommendation

If your primary goal is longer retention, I'd recommend keeping the T-Pot Hive deployment as close to stock as possible and using a separate Elasticsearch cluster for long-term storage and analytics.

For example:

Sensors
    │
    ▼
 T-Pot Hive
    │
    ├── Local Elasticsearch (short retention)
    │
    └── Logstash / pipeline
            │
            ▼
  Separate Elasticsearch cluster
      (hot/warm ILM)

This approach has a few advantages:

• T-Pot remains easy to upgrade.
• Elasticsearch can scale independently.
• ILM, hot/warm tiers, and retention policies can evolve without modifying the T-Pot stack.
• Failures in one environment are less likely to affect the other.

If you do decide to cluster the embedded Elasticsearch instead, expect to maintain those Docker Compose and Elasticsearch configuration changes yourself, as future T-Pot updates may require reapplying them.

If this solves your problem, feel free to mark it as the accepted answer so others can find it easily.