#grpc #envoyproxy #grpc-web
Вопрос:
У меня есть веб-приложение SPA, которое использует gRPC web и посланника для прокси-сервера обратно на сервер, который говорит на gRPC. Все это отлично работает, никаких проблем.
Я пытаюсь сделать это безопасным с помощью HTTPS/TLS и просто продолжаю сталкиваться с проблемами и не могу заставить это работать.
Наша установка такова:
Web Client SPA app (accessed from web node.js server also running on the lahinch server. URL is https://lahinch.mycorp.com ). Web app connects to the envoy proxy using this address "https://coxos.mycorp.COM:8090"
Envoy Proxy (coxos - 172.16.0.116) - listens on port 8090 and proxies to port 50251
Backend gRPC server (lahinch - 172.16.0.109) - listens on port 50251
Из чтения документов envoy следует, что веб-клиент находится ниже по потоку, а серверный сервер-выше по потоку.
Вот мой посланник.файл yaml
admin:
access_log_path: /tmp/admin_access.log
address:
socket_address:
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 8090
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
'@type': >-
type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
access_log:
- name: envoy.access_loggers.file
typed_config:
'@type': >-
type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
path: /dev/stdout
- name: envoy.access_loggers.http_grpc
typed_config:
'@type': >-
type.googleapis.com/envoy.extensions.access_loggers.grpc.v3.HttpGrpcAccessLogConfig
common_config:
log_name: envoygrpclog
grpc_service:
envoy_grpc:
cluster_name: controlweb_backendservice
transport_api_version: V3
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains:
- '*'
routes:
- match:
prefix: /
route:
cluster: controlweb_backendservice
hash_policy:
- header:
header_name: x-session-hash
max_stream_duration:
grpc_timeout_header_max: 300s
cors:
allow_origin_string_match:
- safe_regex:
google_re2: {}
regex: .*
allow_methods: 'GET, PUT, DELETE, POST, OPTIONS'
allow_headers: >-
keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,grpc-status-details-bin,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout,access-token,x-session-hash
expose_headers: >-
grpc-status-details-bin,grpc-status,grpc-message,access-token
max_age: '1728000'
http_filters:
- name: envoy.filters.http.grpc_web
typed_config:
'@type': >-
type.googleapis.com/envoy.extensions.filters.http.grpc_web.v3.GrpcWeb
- name: envoy.filters.http.cors
typed_config:
'@type': >-
type.googleapis.com/envoy.extensions.filters.http.cors.v3.Cors
- name: envoy.filters.http.router
typed_config:
'@type': >-
type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
# https://www.envoyproxy.io/docs/envoy/v1.17.0/api-v3/extensions/transport_sockets/tls/v3/tls.proto#extensions-transport-sockets-tls-v3-downstreamtlscontext
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
# Certificate must be PEM-encoded
filename: /etc/lahinch.pem
private_key:
filename: /etc/lahinch.key.pem
validation_context:
trusted_ca:
filename: /etc/ssl/certs/ZZZ-CA256.pem
clusters:
- name: controlweb_backendservice
type: LOGICAL_DNS
connect_timeout: 0.25s
dns_lookup_family: V4_ONLY
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: cluster_controlweb_backendservice
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: lahinch.mycorp.com
port_value: 50251
http2_protocol_options: {}
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
'@type': >-
type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
filename: /etc/lahinch.pem
private_key:
filename: /etc/lahinch.key.pem
validation_context:
trusted_ca:
filename: /etc/ssl/certs/ZZZ-CA256.pem
Используя это, я получаю следующее в журнале посланника, когда пытаюсь запустить свое веб-приложение:
[2021-04-09 22:08:33.939][17][debug][conn_handler] [source/server/connection_handler_impl.cc:501] [C2] new connection
[2021-04-09 22:08:33.945][17][debug][http] [source/common/http/conn_manager_impl.cc:254] [C2] new stream
[2021-04-09 22:08:33.945][17][debug][http] [source/common/http/conn_manager_impl.cc:886] [C2][S3055347406573314092] request headers complete (end_stream=false):
':authority', 'coxos.mycorp.com:8090'
':path', '/WanderAuth.HostService/LogIn'
':method', 'POST'
'connection', 'keep-alive'
'content-length', '124'
'accept', 'application/grpc-web-text'
'x-user-agent', 'grpc-web-javascript/0.1'
'access-token', ''
'x-grpc-web', '1'
'user-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36 Edg/89.0.774.68'
'grpc-timeout', '90000m'
'content-type', 'application/grpc-web-text'
'origin', 'https://lahinch.mycorp.com'
'sec-fetch-site', 'same-site'
'sec-fetch-mode', 'cors'
'sec-fetch-dest', 'empty'
'referer', 'https://lahinch.mycorp.com/'
'accept-encoding', 'gzip, deflate, br'
'accept-language', 'en-US,en;q=0.9'
[2021-04-09 22:08:33.946][17][debug][router] [source/common/router/router.cc:425] [C2][S3055347406573314092] cluster 'controlweb_backendservice' match for URL '/WanderAuth.HostService/LogIn'
[2021-04-09 22:08:33.946][17][debug][router] [source/common/router/router.cc:582] [C2][S3055347406573314092] router decoding headers:
':authority', 'coxos.mycorp.com:8090'
':path', '/WanderAuth.HostService/LogIn'
':method', 'POST'
':scheme', 'https'
'accept', 'application/grpc-web-text'
'x-user-agent', 'grpc-web-javascript/0.1'
'access-token', ''
'x-grpc-web', '1'
'user-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36 Edg/89.0.774.68'
'grpc-timeout', '90000m'
'content-type', 'application/grpc'
'origin', 'https://lahinch.mycorp.com'
'sec-fetch-site', 'same-site'
'sec-fetch-mode', 'cors'
'sec-fetch-dest', 'empty'
'referer', 'https://lahinch.mycorp.com/'
'accept-encoding', 'gzip, deflate, br'
'accept-language', 'en-US,en;q=0.9'
'x-forwarded-proto', 'https'
'x-request-id', 'a4a041ab-dc29-4ed7-a342-90ac03b3be3c'
'te', 'trailers'
'grpc-accept-encoding', 'identity'
'x-envoy-expected-rq-timeout-ms', '15000'
[2021-04-09 22:08:33.946][17][debug][pool] [source/common/http/conn_pool_base.cc:79] queueing stream due to no available connections
[2021-04-09 22:08:33.946][17][debug][pool] [source/common/conn_pool/conn_pool_base.cc:106] creating a new connection
[2021-04-09 22:08:33.946][17][debug][client] [source/common/http/codec_client.cc:41] [C3] connecting
[2021-04-09 22:08:33.946][17][debug][connection] [source/common/network/connection_impl.cc:860] [C3] connecting to 172.16.0.109:50251
[2021-04-09 22:08:33.946][17][debug][connection] [source/common/network/connection_impl.cc:876] [C3] connection in progress
[2021-04-09 22:08:33.946][17][debug][http2] [source/common/http/http2/codec_impl.cc:1184] [C3] updating connection-level initial window size to 268435456
[2021-04-09 22:08:33.946][17][debug][http] [source/common/http/filter_manager.cc:755] [C2][S3055347406573314092] request end stream
[2021-04-09 22:08:33.947][17][debug][connection] [source/common/network/connection_impl.cc:666] [C3] connected
[2021-04-09 22:08:33.947][17][debug][connection] [source/extensions/transport_sockets/tls/ssl_socket.cc:224] [C3] TLS error: 268435703:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER
[2021-04-09 22:08:33.948][17][debug][connection] [source/common/network/connection_impl.cc:241] [C3] closing socket: 0
[2021-04-09 22:08:33.948][17][debug][connection] [source/extensions/transport_sockets/tls/ssl_socket.cc:224] [C3] TLS error: 268435703:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER
[2021-04-09 22:08:33.948][17][debug][client] [source/common/http/codec_client.cc:99] [C3] disconnect. resetting 0 pending requests
[2021-04-09 22:08:33.948][17][debug][pool] [source/common/conn_pool/conn_pool_base.cc:343] [C3] client disconnected, failure reason: TLS error: 268435703:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER
[2021-04-09 22:08:33.948][17][debug][router] [source/common/router/router.cc:1026] [C2][S3055347406573314092] upstream reset: reset reason: connection failure, transport failure reason: TLS error: 268435703:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER
[2021-04-09 22:08:33.948][17][debug][http] [source/common/http/filter_manager.cc:839] [C2][S3055347406573314092] Sending local reply with details upstream_reset_before_response_started{connection failure,TLS error: 268435703:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER}
[2021-04-09 22:08:33.948][17][debug][http] [source/common/http/conn_manager_impl.cc:1484] [C2][S3055347406573314092] encoding headers via codec (end_stream=false):
':status', '503'
'content-length', '190'
'content-type', 'application/grpc-web-text proto'
'access-control-allow-origin', 'https://lahinch.mycorp.com'
'access-control-expose-headers', 'grpc-status-details-bin,grpc-status,grpc-message,access-token'
'date', 'Fri, 09 Apr 2021 22:08:33 GMT'
'server', 'envoy'
[2021-04-09 22:08:36.139][9][debug][upstream] [source/common/upstream/logical_dns_cluster.cc:101] starting async DNS resolution for lahinch.mycorp.com
[2021-04-09 22:08:36.139][9][debug][main] [source/server/server.cc:199] flushing stats
[2021-04-09 22:08:36.141][9][debug][upstream] [source/common/upstream/logical_dns_cluster.cc:109] async DNS resolution complete for lahinch.mycorp.com
[2021-04-09 22:08:36.141][9][debug][upstream] [source/common/upstream/logical_dns_cluster.cc:155] DNS refresh rate reset for lahinch.mycorp.com, refresh rate 5000 ms
Таким образом, ошибка выглядит следующим образом: Ошибка TLS: 268435703:Процедуры SSL:OPENSSL_internal:НЕПРАВИЛЬНЫЙ НОМЕР ВЕРСИИ
Я просмотрел эту ошибку, и, похоже, она связана с безопасностью и сертификатами. Но я не смог найти хорошего ответа на вопрос, что я делаю не так.
Когда дело доходит до требуемых сертификатов, должны ли сертификаты совпадать с теми, которые используются клиентом(нижестоящим), прокси-сервером или серверной частью(вышестоящий сервер), или и тем, и другим? Я пробовал использовать разные сертификаты для разных серверов и одни и те же сертификаты для серверов, и я все равно получаю ту же ошибку.
Комментарии:
1. Здесь приведен пример использования grpc-web с TLS: github.com/salrashid123/grpc_web_with_gke . Там есть раздел о том, как использовать сертификаты. Вам не нужно использовать GKE, если вам это не нужно.