스프링 MVC는 왜 스레드를 최대 200개까지 사용할까

// 서블릿 컨테이너의 동작 방식
class ServletContainer {
    ThreadPool pool = new ThreadPool(200); // 미리 생성

    void handleRequest(HttpRequest request) {
        Thread thread = pool.getThread(); // 풀에서 가져옴
        thread.run(() -> {
            servlet.service(request);
        });
        pool.returnThread(thread); // 다시 반환
    }
}

// 서블릿 라이프사이클
public class MyServlet extends HttpServlet {

    // 1. init(): 서블릿이 처음 생성될 때 단 한 번 호출
    @Override
    public void init(ServletConfig config) throws ServletException {
        System.out.println("서블릿 초기화!");
        // DB 커넥션 풀 초기화 등의 작업
    }

    // 2. service(): 요청이 올 때마다 호출 (멀티스레드로 동작)
    @Override
    protected void service(HttpServletRequest req, HttpServletResponse res)
            throws ServletException, IOException {
        // GET, POST 등 HTTP 메서드에 따라 분기
        String method = req.getMethod();
        if (method.equals("GET")) {
            doGet(req, res);
        } else if (method.equals("POST")) {
            doPost(req, res);
        }
    }

    // 3. destroy(): 서블릿이 제거될 때 단 한 번 호출
    @Override
    public void destroy() {
        System.out.println("서블릿 종료!");
        // 리소스 정리
    }
}

// 첫 번째 요청
GET http://localhost:8080/myServlet
// 응답 시간: 38.50ms (서블릿 초기화 포함)

// 두 번째 요청
GET http://localhost:8080/myServlet
// 응답 시간: 6.61ms (서블릿 재사용!)

// 위험한 코드!
public class DangerousServlet extends HttpServlet {
    private int count = 0; // 인스턴스 변수 (공유됨!)

    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse res)
            throws ServletException, IOException {
        count++; // 여러 스레드가 동시에 접근!
        res.getWriter().write("Count: " + count);
    }
}

// 시나리오:
// Thread 1: count++ (0 → 1)
// Thread 2: count++ (동시에 접근, 1 → 2가 아니라 1 → 1이 될 수도!)
// Thread 3: count++ (경쟁 조건 발생!)

public class SafeServlet extends HttpServlet {
    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse res)
            throws ServletException, IOException {
        int count = 0; // 지역 변수 (스레드마다 Stack에 따로 생김)
        count++;
        res.getWriter().write("Count: " + count);
    }
}

public class SynchronizedServlet extends HttpServlet {
    private int count = 0;

    @Override
    protected synchronized void doGet(HttpServletRequest req, HttpServletResponse res)
            throws ServletException, IOException {
        count++; // 한 번에 하나의 스레드만 접근
        res.getWriter().write("Count: " + count);
    }
}

// 스프링의 핵심: DispatcherServlet
public class DispatcherServlet extends FrameworkServlet {

    @Override
    protected void doService(HttpServletRequest req, HttpServletResponse res)
            throws Exception {

        // 1. 핸들러 매핑: 어떤 컨트롤러가 처리할지 찾기
        HandlerExecutionChain handler = getHandler(req);
        // GET /api/user → UserController.getUser()

        // 2. 핸들러 어댑터: 컨트롤러 실행 방법 결정
        HandlerAdapter adapter = getHandlerAdapter(handler);
        // @RestController면 JSON 반환
        // @Controller면 View 이름 반환

        // 3. 실제 실행
        ModelAndView mv = adapter.handle(req, res, handler);

        // 4. 뷰 렌더링 (필요한 경우)
        render(mv, req, res);
    }
}

메모리: 10,000 * 2MB = 20GB
컨텍스트 스위칭: 초당 수백만 번
CPU: 스레드 전환만 하다가 죽음

// NIO Connector의 동작 방식
class NioConnector {
    Selector selector = Selector.open();
    ThreadPool workerPool = new ThreadPool(200);

    void acceptConnection(SocketChannel channel) {
        // Selector에 등록만 하고 스레드는 할당 안 함
        channel.register(selector, SelectionKey.OP_READ);
    }

    void pollLoop() {
        while (true) {
            // 여러 연결을 동시에 감시
            selector.select();

            for (SelectionKey key : selector.selectedKeys()) {
                if (key.isReadable()) {
                    // 데이터가 있을 때만 워커 스레드 할당
                    SocketChannel ch = (SocketChannel) key.channel();
                    workerPool.submit(() -> processRequest(ch));
                }
            }
        }
    }
}

BIO:
200개 스레드 = 최대 200개 동시 연결

NIO:
200개 워커 스레드
+ Poller 스레드 2개
= 최대 8,192개 동시 연결 (Tomcat 8 기본값)

// Tomcat 8.5부터
<Connector protocol="HTTP/1.1" />
// → 자동으로 NIO 사용

// BIO를 명시적으로 설정해도
<Connector protocol="org.apache.coyote.http11.Http11Protocol" />
// → 경고 로그와 함께 NIO로 전환됨

http-nio-8080-Acceptor-0 (1개)
  → 연결 수락

http-nio-8080-ClientPoller-0 (2개)
http-nio-8080-ClientPoller-1
  → Selector로 연결 감시

http-nio-8080-exec-1 (10~200개)
http-nio-8080-exec-2
...
http-nio-8080-exec-200
  → 실제 요청 처리

1. max-connections (8192개) 내의 연결은 Poller가 관리
2. accept-count (100개) 까지는 OS 레벨 큐에서 대기
3. 두 제한을 모두 초과하면 connection timeout

스레드가 sleep 상태일 때: 약 16KB (물리 RAM)
스택을 실제로 쓸 때: 최대 1MB까지 증가

스택 메모리:
- 최소: 200 * 16KB = 3.2MB
- 최대: 200 * 1MB = 200MB

전체 JVM:
- Heap: 예를 들어 2GB (-Xmx2g)
- Stack: 200MB (200 threads)
- Metaspace: 약 256MB
= 총 2.5GB 정도

스레드 A 레지스터 저장 → 스레드 B 레지스터 복원 → TLB flush
= 약 1-2μs

4 / 200 = 0.02 (2%)

4 / 2000 = 0.002 (0.2%)
CPU가 스레드 전환만 하다가 끝남

CPU: Pentium III 500MHz
RAM: 128-512MB
동시 접속자: 수백 명 수준

300개 동시 요청: 정상 처리
10,000개 요청: timeout 발생

15,000개 동시 요청: 정상 처리

threads.max=50
→ 서버 리소스는 남는데 클라이언트는 대기
→ TPS 낮아짐

threads.max=2000
→ 메모리 부족 (2000 * 1MB = 2GB)
→ 컨텍스트 스위칭 과다
→ CPU는 100%인데 처리량은 낮음

1. 애플리케이션 로직 복잡도 (CPU 사용률)
2. I/O 대기 시간 (DB, API 호출)
3. 예상 트래픽
4. 하드웨어 스펙 (코어 수, RAM)
5. DBCP 크기 (DB 커넥션 풀)

스레드 풀 크기 = CPU 코어 수 × (1 + 대기 시간 / 처리 시간)

CPU: 4코어
평균 처리 시간: 100ms
평균 I/O 대기: 900ms (DB 쿼리)

스레드 풀 크기 = 4 × (1 + 900/100)
               = 4 × 10
               = 40

server:
  tomcat:
    threads:
      max: 200          # 최대 워커 스레드
      min-spare: 10     # 최소 유지 스레드
    accept-count: 100   # 대기 큐 크기
    max-connections: 8192  # 최대 동시 연결

Spring Boot 2.3 이후: server.tomcat.threads.max
Spring Boot 2.3 이전: server.tomcat.max-threads

java.lang.OutOfMemoryError: unable to create new native thread

최대 스레드 수 = 가용 메모리 / (스택 크기 * 1024 * 1024)

서버 메모리: 4GB
JVM Heap: 2GB (-Xmx2g)
남은 메모리: 2GB
스레드 스택 크기: 1MB (-Xss1m)

이론적 최대 스레드: 2048개
실제 제한: ulimit -u (예: 1024)
→ 1024개까지만 생성 가능

// 서버 터트리기 (절대 따라하지 마세요)
class ThreadBomb {
    public static void main(String[] args) {
        int count = 0;
        try {
            while (true) {
                new Thread(() -> {
                    try {
                        Thread.sleep(Long.MAX_VALUE);
                    } catch (InterruptedException e) {
                    }
                }).start();
                count++;
                System.out.println("Threads: " + count);
            }
        } catch (OutOfMemoryError e) {
            System.err.println("터졌다! " + count + "개 스레드에서 사망");
            // Threads: 12543
            // java.lang.OutOfMemoryError: unable to create new native thread
        }
    }
}

톰캣 설정:
- threads.max: 200
- accept-count: 100
- max-connections: 8192

상황:
1. 200개 스레드 전부 DB 쿼리 대기 중 (각 10초씩)
2. accept-count 큐도 100개 꽉 참
3. 301번째 요청부터는 connection refused

org.apache.tomcat.util.threads.ThreadPoolExecutor
All threads (200) are currently busy

@RestController
class StreamController {

    @GetMapping("/api/stream/{id}")
    public StreamResponse getStream(@PathVariable Long id) {
        // DB 조회 - 평균 100ms
        Stream stream = streamRepository.findById(id);

        // 외부 API 호출 - 평균 500ms
        User user = oauthClient.getUserInfo(stream.userId);

        // Redis 조회 - 평균 10ms
        ViewCount views = redisTemplate.get(stream.id);

        return new StreamResponse(stream, user, views);
    }
}

1. 200개 스레드가 전부 OAuth 대기
2. 신규 요청은 큐에서 대기
3. 큐마저 꽉 차면 connection refused
4. 사용자는 "서버 점검 중" 페이지만 봄

java.lang.OutOfMemoryError: Java heap space

// 위험한 코드
class UserContext {
    private static final ThreadLocal<User> CURRENT_USER =
        new ThreadLocal<>();

    public static void setUser(User user) {
        CURRENT_USER.set(user);
        // remove() 안 하면 메모리 누수!
    }
}

@RestController
class UserController {

    @GetMapping("/api/user")
    public UserResponse getUser() {
        User user = userService.findUser();
        UserContext.setUser(user); // 설정만 하고
        return new UserResponse(user);
        // remove() 안 함 → 스레드가 재사용될 때 남아있음
    }
}

// 위험한 코드
@Service
class EventProcessor {
    private final Queue<Event> queue =
        new LinkedBlockingQueue<>(); // 크기 제한 없음!

    @Async
    public void processEvent(Event event) {
        queue.offer(event);
    }
}

@RestController
class UserController {

    @GetMapping("/api/user")
    public UserResponse getUser() {
        try {
            User user = userService.findUser();
            UserContext.setUser(user);
            return new UserResponse(user);
        } finally {
            UserContext.remove(); // 꼭 정리!
        }
    }
}

@Configuration
class AsyncConfig {
    @Bean
    public Executor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setQueueCapacity(1000); // 크기 제한!
        executor.setRejectedExecutionHandler(
            new ThreadPoolExecutor.CallerRunsPolicy()
        );
        return executor;
    }
}

요청 관련:
- requestCount: 처리된 총 요청 수
- processingTime: 누적 처리 시간
- maxTime: 최대 처리 시간

스레드 관련:
- currentThreadsBusy: 현재 사용 중인 스레드
- maxThreads: 최대 스레드 수

리소스 관련:
- HeapMemoryUsage: JVM 메모리 사용량
- CollectionCount: GC 횟수

I/O 작업 많으면 (DB 쿼리, API 호출):
→ 스레드 늘려도 됨 (500~1000)

CPU 작업 많으면 (이미지 처리, 암호화):
→ 코어 수 * 2 정도로 제한

동시 접속 만 명 이상:
→ Virtual Threads 고려 (Java 21+)

// How a servlet container works
class ServletContainer {
    ThreadPool pool = new ThreadPool(200); // Pre-created

    void handleRequest(HttpRequest request) {
        Thread thread = pool.getThread(); // Get from pool
        thread.run(() -> {
            servlet.service(request);
        });
        pool.returnThread(thread); // Return to pool
    }
}

// Servlet lifecycle
public class MyServlet extends HttpServlet {

    // 1. init(): Called once when the servlet is first created
    @Override
    public void init(ServletConfig config) throws ServletException {
        System.out.println("서블릿 초기화!");
        // Tasks like DB connection pool initialization
    }

    // 2. service(): Called on every request (operates in multithreaded fashion)
    @Override
    protected void service(HttpServletRequest req, HttpServletResponse res)
            throws ServletException, IOException {
        // Branch based on HTTP method
        String method = req.getMethod();
        if (method.equals("GET")) {
            doGet(req, res);
        } else if (method.equals("POST")) {
            doPost(req, res);
        }
    }

    // 3. destroy(): Called once when the servlet is removed
    @Override
    public void destroy() {
        System.out.println("서블릿 종료!");
        // Resource cleanup
    }
}

// First request
GET http://localhost:8080/myServlet
// Response time: 38.50ms (includes servlet initialization)

// Second request
GET http://localhost:8080/myServlet
// Response time: 6.61ms (servlet reused!)

// Dangerous code!
public class DangerousServlet extends HttpServlet {
    private int count = 0; // Instance variable (shared!)

    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse res)
            throws ServletException, IOException {
        count++; // Multiple threads access simultaneously!
        res.getWriter().write("Count: " + count);
    }
}

// Scenario:
// Thread 1: count++ (0 -> 1)
// Thread 2: count++ (concurrent access, could become 1 -> 1 instead of 1 -> 2!)
// Thread 3: count++ (race condition!)

public class SafeServlet extends HttpServlet {
    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse res)
            throws ServletException, IOException {
        int count = 0; // Local variable (created separately in each thread's Stack)
        count++;
        res.getWriter().write("Count: " + count);
    }
}

public class SynchronizedServlet extends HttpServlet {
    private int count = 0;

    @Override
    protected synchronized void doGet(HttpServletRequest req, HttpServletResponse res)
            throws ServletException, IOException {
        count++; // Only one thread accesses at a time
        res.getWriter().write("Count: " + count);
    }
}

// The core of Spring: DispatcherServlet
public class DispatcherServlet extends FrameworkServlet {

    @Override
    protected void doService(HttpServletRequest req, HttpServletResponse res)
            throws Exception {

        // 1. Handler mapping: find which controller handles this
        HandlerExecutionChain handler = getHandler(req);
        // GET /api/user -> UserController.getUser()

        // 2. Handler adapter: determine how to execute the controller
        HandlerAdapter adapter = getHandlerAdapter(handler);
        // @RestController returns JSON
        // @Controller returns a View name

        // 3. Actual execution
        ModelAndView mv = adapter.handle(req, res, handler);

        // 4. View rendering (if needed)
        render(mv, req, res);
    }
}

Memory: 10,000 * 2MB = 20GB
Context switching: millions of times per second
CPU: dies just from thread switching

// How the NIO Connector works
class NioConnector {
    Selector selector = Selector.open();
    ThreadPool workerPool = new ThreadPool(200);

    void acceptConnection(SocketChannel channel) {
        // Just register with the Selector; no thread assigned
        channel.register(selector, SelectionKey.OP_READ);
    }

    void pollLoop() {
        while (true) {
            // Monitor multiple connections simultaneously
            selector.select();

            for (SelectionKey key : selector.selectedKeys()) {
                if (key.isReadable()) {
                    // Assign a worker thread only when data is available
                    SocketChannel ch = (SocketChannel) key.channel();
                    workerPool.submit(() -> processRequest(ch));
                }
            }
        }
    }
}

BIO:
200 threads = max 200 concurrent connections

NIO:
200 worker threads
+ 2 Poller threads
= max 8,192 concurrent connections (Tomcat 8 default)

// From Tomcat 8.5
<Connector protocol="HTTP/1.1" />
// -> Automatically uses NIO

// Even if you explicitly set BIO
<Connector protocol="org.apache.coyote.http11.Http11Protocol" />
// -> Switches to NIO with a warning log

http-nio-8080-Acceptor-0 (1)
  -> Accepts connections

http-nio-8080-ClientPoller-0 (2)
http-nio-8080-ClientPoller-1
  -> Monitors connections via Selector

http-nio-8080-exec-1 (10~200)
http-nio-8080-exec-2
...
http-nio-8080-exec-200
  -> Processes actual requests

1. Connections within max-connections (8192) are managed by the Poller
2. Up to accept-count (100) wait in the OS-level queue
3. When both limits are exceeded, connection timeout occurs

Thread in sleep state: ~16KB (physical RAM)
When stack is actively used: up to 1MB

Stack memory:
- Minimum: 200 * 16KB = 3.2MB
- Maximum: 200 * 1MB = 200MB

Total JVM:
- Heap: e.g., 2GB (-Xmx2g)
- Stack: 200MB (200 threads)
- Metaspace: ~256MB
= About 2.5GB total

Save Thread A registers -> Restore Thread B registers -> TLB flush
= ~1-2 us

4 / 200 = 0.02 (2%)

4 / 2000 = 0.002 (0.2%)
CPU spends all its time just switching threads

CPU: Pentium III 500MHz
RAM: 128-512MB
Concurrent users: hundreds

300 concurrent requests: processed normally
10,000 requests: timeouts occurred

15,000 concurrent requests: processed normally

threads.max=50
-> Server resources are idle while clients wait
-> TPS decreases

threads.max=2000
-> Memory shortage (2000 * 1MB = 2GB)
-> Excessive context switching
-> CPU at 100% but throughput is low

1. Application logic complexity (CPU usage)
2. I/O wait time (DB, API calls)
3. Expected traffic
4. Hardware specs (number of cores, RAM)
5. DBCP size (DB connection pool)

Thread pool size = CPU cores x (1 + wait time / processing time)

CPU: 4 cores
Average processing time: 100ms
Average I/O wait: 900ms (DB query)

Thread pool size = 4 x (1 + 900/100)
               = 4 x 10
               = 40

server:
  tomcat:
    threads:
      max: 200          # Maximum worker threads
      min-spare: 10     # Minimum idle threads
    accept-count: 100   # Wait queue size
    max-connections: 8192  # Maximum concurrent connections

Spring Boot 2.3+: server.tomcat.threads.max
Spring Boot before 2.3: server.tomcat.max-threads

java.lang.OutOfMemoryError: unable to create new native thread

Maximum threads = Available memory / (Stack size * 1024 * 1024)

Server memory: 4GB
JVM Heap: 2GB (-Xmx2g)
Remaining memory: 2GB
Thread stack size: 1MB (-Xss1m)

Theoretical maximum threads: 2048
Actual limit: ulimit -u (e.g., 1024)
-> Can only create up to 1024

// Crashing the server (do NOT try this!)
class ThreadBomb {
    public static void main(String[] args) {
        int count = 0;
        try {
            while (true) {
                new Thread(() -> {
                    try {
                        Thread.sleep(Long.MAX_VALUE);
                    } catch (InterruptedException e) {
                    }
                }).start();
                count++;
                System.out.println("Threads: " + count);
            }
        } catch (OutOfMemoryError e) {
            System.err.println("터졌다! " + count + "개 스레드에서 사망");
            // Threads: 12543
            // java.lang.OutOfMemoryError: unable to create new native thread
        }
    }
}

Tomcat settings:
- threads.max: 200
- accept-count: 100
- max-connections: 8192

Situation:
1. All 200 threads waiting on DB queries (10 seconds each)
2. accept-count queue is also full at 100
3. From the 301st request onward: connection refused

org.apache.tomcat.util.threads.ThreadPoolExecutor
All threads (200) are currently busy

@RestController
class StreamController {

    @GetMapping("/api/stream/{id}")
    public StreamResponse getStream(@PathVariable Long id) {
        // DB query - average 100ms
        Stream stream = streamRepository.findById(id);

        // External API call - average 500ms
        User user = oauthClient.getUserInfo(stream.userId);

        // Redis query - average 10ms
        ViewCount views = redisTemplate.get(stream.id);

        return new StreamResponse(stream, user, views);
    }
}

1. All 200 threads waiting on OAuth
2. New requests wait in queue
3. When the queue is also full: connection refused
4. Users only see a "Server maintenance" page

java.lang.OutOfMemoryError: Java heap space

// Dangerous code
class UserContext {
    private static final ThreadLocal<User> CURRENT_USER =
        new ThreadLocal<>();

    public static void setUser(User user) {
        CURRENT_USER.set(user);
        // Not calling remove() causes memory leak!
    }
}

@RestController
class UserController {

    @GetMapping("/api/user")
    public UserResponse getUser() {
        User user = userService.findUser();
        UserContext.setUser(user); // Set but
        return new UserResponse(user);
        // remove() not called -> stays when thread is reused
    }
}

// Dangerous code
@Service
class EventProcessor {
    private final Queue<Event> queue =
        new LinkedBlockingQueue<>(); // No size limit!

    @Async
    public void processEvent(Event event) {
        queue.offer(event);
    }
}

@RestController
class UserController {

    @GetMapping("/api/user")
    public UserResponse getUser() {
        try {
            User user = userService.findUser();
            UserContext.setUser(user);
            return new UserResponse(user);
        } finally {
            UserContext.remove(); // Always clean up!
        }
    }
}

@Configuration
class AsyncConfig {
    @Bean
    public Executor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setQueueCapacity(1000); // Size limit!
        executor.setRejectedExecutionHandler(
            new ThreadPoolExecutor.CallerRunsPolicy()
        );
        return executor;
    }
}

Request-related:
- requestCount: total requests processed
- processingTime: cumulative processing time
- maxTime: maximum processing time

Thread-related:
- currentThreadsBusy: currently active threads
- maxThreads: maximum thread count

Resource-related:
- HeapMemoryUsage: JVM memory usage
- CollectionCount: GC count

Heavy I/O operations (DB queries, API calls):
-> It's okay to increase threads (500~1000)

Heavy CPU operations (image processing, encryption):
-> Limit to about core count * 2

10,000+ concurrent connections:
-> Consider Virtual Threads (Java 21+)