EC2 SSH Connection Refused? The Hidden Network Trap Behind AWS Security Groups

INTRODUCTION

While working on a massive infrastructure modernization project for a secure FinTech platform, we were tasked with locking down access to internal compute resources. The mandate was strict: no public IP addresses, no open inbound ports, and all access must be tightly controlled and audited.

During the setup of a new fleet of hardened AWS EC2 instances, a senior engineer flagged a strange networking anomaly. The instances were deliberately provisioned with no inbound rules in their attached Security Groups. By standard AWS networking principles, any inbound SSH connection attempt should have resulted in a silent packet drop, leading to a Timeout error. Instead, the terminal immediately threw a Connection Refused error.

This subtle discrepancy sparked an immediate investigation. In high-security environments, unexpected network responses can indicate routing misconfigurations, rogue intermediaries, or compromised networking layers. We realized that this specific behavior is a common source of confusion for engineering teams migrating to the cloud. This challenge inspired this article to help other teams diagnose false cloud networking symptoms and properly architect their remote access strategies.

PROBLEM CONTEXT

To understand why this anomaly was alarming, we must first look at how AWS Security Groups and standard Transmission Control Protocol (TCP) handshakes operate.

AWS Security Groups act as stateful, instance-level firewalls. When you attempt to connect to an EC2 instance, the network packet must pass through the Security Group before it reaches the instance’s operating system.

• Timeout: If a Security Group lacks an inbound rule allowing the traffic, AWS silently drops the TCP SYN packet. The client waits for an acknowledgment (SYN-ACK) that never comes, eventually timing out.
• Connection Refused: This happens when the TCP SYN packet reaches a destination, but the destination actively rejects it by sending back an RST (Reset) packet. This typically occurs if the firewall allows the traffic, but the underlying service (like the SSH daemon) is down or not listening on that port.

In our FinTech scenario, the EC2 instance had absolutely no inbound rules. Therefore, it was physically impossible for the EC2 instance to send an RST packet back, because the Security Group was dropping the initial request. Yet, the terminal clearly showed Connection Refused. This paradox highlighted the importance of deep network diagnostics, a skill set we prioritize when organizations look to hire software developer teams capable of handling complex cloud deployments.

WHAT WENT WRONG

To get to the bottom of the issue, we began by reproducing the error. Running a standard SSH command yielded an immediate rejection:

$ ssh -i private-key.pem ec2-user@10.x.x.x
ssh: connect to host 10.x.x.x port 22: Connection refused

The speed of the failure was the first clue. A true timeout takes time—usually 30 to 60 seconds depending on the client configuration. An immediate “Connection Refused” means an entity in the network path actively intercepted the request and terminated it.

We ran the SSH command in verbose mode to inspect the connection lifecycle:

$ ssh -vvv -i private-key.pem ec2-user@10.x.x.x
OpenSSH_8.9p1, OpenSSL 1.1.1f  31 Mar 2020
debug1: Reading configuration data /etc/ssh/ssh_config
debug2: resolving "10.x.x.x" port 22
debug3: ssh_connect_direct: entering
debug1: Connecting to 10.x.x.x [10.x.x.x] port 22.
debug1: connect to address 10.x.x.x port 22: Connection refused

The logs confirmed the rapid rejection but did not pinpoint the source of the RST packet. Was it a local misconfiguration? A corporate VPN rule? Or an AWS networking glitch?

HOW WE APPROACHED THE SOLUTION

When you encounter a networking paradox, the best approach is to drop down to the packet level. If AWS wasn’t sending the RST packet, we needed to find out who was.

Step 1: Packet Capture Analysis

We ran tcpdump on the local developer machine to monitor the exact TCP handshake sequence during the SSH attempt:

$ sudo tcpdump -i en0 host 10.x.x.x

The output revealed a critical piece of information:

14:02:11.123 IP local-machine.54321 > 10.x.x.x.22: Flags [S], seq 123456...
14:02:11.125 IP 10.x.x.x.22 > local-machine.54321: Flags [R.], seq 0, ack 123457...

Notice the timestamp. The RST (Reset) packet arrived just 2 milliseconds after the SYN packet was sent. The EC2 instance was hosted in an AWS region with a known baseline latency of 40-50 milliseconds from the client’s location. A 2-millisecond round trip meant the packet never left the local network.

Step 2: Identifying the Intermediary

By analyzing the Time-To-Live (TTL) of the returning RST packet, we deduced that the connection was being intercepted exactly one network hop away. The developer was connected to a strict corporate VPN configured for the FinTech environment.

The corporate egress firewall had a strict rule: all outbound port 22 (SSH) traffic originating from developer workstations was explicitly blocked. Instead of silently dropping unauthorized outbound traffic, the corporate firewall was configured to immediately send a TCP Reset (RST) packet back to the client to close the connection efficiently.

The mystery was solved. The developer assumed AWS was refusing the connection, but in reality, their own corporate network was rejecting the outbound request before it ever reached the cloud.

FINAL IMPLEMENTATION

While the mystery of the “Connection Refused” error was resolved, the business requirement remained: developers needed secure access to these hardened instances to perform deployment validations and debugging.

Opening port 22—even internally over a VPN—was an anti-pattern we wanted to eliminate. This is a common architectural pivot we implement when companies hire aws developers for secure cloud infrastructure.

Migrating to AWS Systems Manager (SSM)

We bypassed the need for traditional SSH entirely by implementing AWS Systems Manager Session Manager. This approach provides secure, auditable, and interactive shell access without requiring open inbound ports or managing SSH keys.

1. IAM Role Configuration

We attached an IAM Instance Profile to the EC2 instances containing the AmazonSSMManagedInstanceCore policy.

2. VPC Endpoint Setup

Since the instances were in private subnets with no internet access, we provisioned VPC Endpoints (PrivateLink) for SSM. This allowed the SSM Agent on the EC2 instances to communicate securely with the Systems Manager control plane over the AWS backbone.

3. Client-Side Access

Developers were instructed to use the AWS CLI with the Session Manager plugin. Access was granted via temporary IAM credentials integrated with the company’s Single Sign-On (SSO) provider.

$ aws ssm start-session --target i-0abcd1234efgh5678

This implementation achieved a zero-trust access model. The instances maintained their completely empty inbound Security Groups, the corporate egress firewall rules remained intact, and developers regained secure access.

LESSONS FOR ENGINEERING TEAMS

Cloud networking issues often present symptoms that point in the wrong direction. Here are the core takeaways engineering leaders should consider, especially when they hire devops engineers for cloud deployments:

• Trust the TCP Handshake: Understand the difference between a dropped packet (Timeout) and an active rejection (Connection Refused). If the symptom contradicts the firewall rules, look for intermediaries.
• Analyze Egress, Not Just Ingress: Developers intuitively blame the destination (AWS Security Groups) when a connection fails. Always verify the outbound (egress) rules of your local network, VPN, or corporate firewall first.
• Utilize Packet Captures: Tools like tcpdump and Wireshark are invaluable. Analyzing packet timing and TTL can quickly determine if a block is happening locally or remotely.
• Eliminate Direct SSH: In modern cloud environments, opening port 22 is rarely necessary. Leverage native cloud management tools like AWS SSM Session Manager or Identity-Aware Proxies.
• Standardize Error Diagnostics: Build runbooks that guide developers through basic connectivity checks (like testing with `nc` or `telnet`) before escalating to infrastructure teams.

WRAP UP

What initially appeared to be a malfunctioning AWS Security Group turned out to be a correctly functioning corporate egress firewall. By diving deep into the packet layer, our team identified the root cause and leveraged the opportunity to modernize the client’s access architecture using AWS SSM.

When migrating complex applications to the cloud, having seasoned engineers who understand both underlying network protocols and cloud-native services is critical. If your organization is facing similar infrastructure challenges or looking to scale your engineering capabilities, contact us to learn how our dedicated remote engineering teams can help secure and streamline your cloud environments.

Social Hashtags

#EC2SSHConnectionRefused #AWS #AWSSecurityGroups #EC2 #CloudSecurity #DevOps #AWSCloud #SSMSessionManager #ZeroTrustSecurity #CloudNetworking #InfrastructureSecurity #AWSDevOps #FinTechSecurity