{"provider_url":"https://hatena.blog","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Ftech.asken.inc%2Fentry%2F20251223\" title=\"SageMaker\u3067vLLM\u3092\u52d5\u304b\u305d\u3046 - asken \u30c6\u30c3\u30af\u30d6\u30ed\u30b0\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","blog_url":"https://tech.asken.inc/","author_name":"techaskencontributor","author_url":"https://blog.hatena.ne.jp/techaskencontributor/","url":"https://tech.asken.inc/entry/20251223","provider_name":"Hatena Blog","image_url":null,"description":"Amazon SageMaker \u3068 vLLM \u3092\u7d44\u307f\u5408\u308f\u305b\u3066\u3001\u9ad8\u901f\u306a LLM \u63a8\u8ad6\u57fa\u76e4\u3092\u69cb\u7bc9\u3059\u308b\u65b9\u6cd5\u3092\u89e3\u8aac\u3057\u307e\u3059\u3002AWS \u306e LMI\uff08Large Model Inference\uff09\u30b3\u30f3\u30c6\u30ca\u3092\u6d3b\u7528\u3057\u3001AsyncLLMEngine \u306b\u3088\u308b\u975e\u540c\u671f\u51e6\u7406\u3001Continuous Batching \u306b\u3088\u308b\u30b9\u30eb\u30fc\u30d7\u30c3\u30c8\u5411\u4e0a\u3001\u30b3\u30fc\u30eb\u30c9\u30b9\u30bf\u30fc\u30c8\u5bfe\u7b56\u307e\u3067\u3001\u672c\u756a\u904b\u7528\u306b\u5fc5\u8981\u306a\u8a2d\u5b9a\u3092\u7db2\u7f85\u3002serving.properties \u3068 model.py \u306e\u66f8\u304d\u65b9\u304b\u3089\u3001SageMaker Endpoint \u306e\u30c7\u30d7\u30ed\u30a4\u307e\u3067\u3001\u30b9\u30c6\u30c3\u30d7\u30d0\u30a4\u30b9\u30c6\u30c3\u30d7\u3067\u8aac\u660e\u3057\u307e\u3059\u3002","published":"2025-12-22 17:00:00","blog_title":"asken \u30c6\u30c3\u30af\u30d6\u30ed\u30b0","width":"100%","version":"1.0","categories":["AI","Advent Calendar","AWS","LLM"],"height":"190","title":"SageMaker\u3067vLLM\u3092\u52d5\u304b\u305d\u3046","type":"rich"}