当前位置 : 首页 » 文章分类 :  开发  »  Apache-HttpClient

Apache-HttpClient

Apache HttpComponents/HttpClient 笔记

Apache HttpComponents
http://hc.apache.org/

httpClient debug 日志

打开 HttpClient 的 debug 日志

<logger name="org.apache.http" level="debug" />

一次完整的 http 请求 debug 日志

2022-08-29 10:15:05.571 [configWatchTaskScheduler-1] DEBUG o.apache.http.client.protocol.RequestAddCookies - CookieSpec selected: default
2022-08-29 10:15:05.571 [configWatchTaskScheduler-1] DEBUG org.apache.http.client.protocol.RequestAuthCache - Auth cache not set in the context
2022-08-29 10:15:05.571 [configWatchTaskScheduler-1] DEBUG o.a.h.i.conn.PoolingHttpClientConnectionManager - Connection request: [route: {}->http://consul-service:8500][total available: 1; route allocated: 2 of 500; total allocated: 2 of 1000]
2022-08-29 10:15:05.572 [configWatchTaskScheduler-1] DEBUG o.a.h.i.conn.PoolingHttpClientConnectionManager - Connection leased: [id: 0][route: {}->http://consul-service:8500][total available: 0; route allocated: 2 of 500; total allocated: 2 of 1000]
2022-08-29 10:15:05.572 [configWatchTaskScheduler-1] DEBUG o.a.h.i.conn.DefaultManagedHttpClientConnection - http-outgoing-0: set socket timeout to 0
2022-08-29 10:15:05.572 [configWatchTaskScheduler-1] DEBUG o.a.h.i.conn.DefaultManagedHttpClientConnection - http-outgoing-0: set socket timeout to 600000
2022-08-29 10:15:05.572 [configWatchTaskScheduler-1] DEBUG org.apache.http.impl.execchain.MainClientExec - Executing request GET /v1/kv/config/tc,online/?recurse&token=acg-2021-ffmumu666&wait=55s&index=546983 HTTP/1.1
2022-08-29 10:15:05.572 [configWatchTaskScheduler-1] DEBUG org.apache.http.impl.execchain.MainClientExec - Target auth state: UNCHALLENGED
2022-08-29 10:15:05.572 [configWatchTaskScheduler-1] DEBUG org.apache.http.impl.execchain.MainClientExec - Proxy auth state: UNCHALLENGED
2022-08-29 10:15:05.572 [configWatchTaskScheduler-1] DEBUG org.apache.http.headers - http-outgoing-0 >> GET /v1/kv/config/tc,online/?recurse&token=acg-2021-ffmumu666&wait=55s&index=546983 HTTP/1.1
2022-08-29 10:15:05.572 [configWatchTaskScheduler-1] DEBUG org.apache.http.headers - http-outgoing-0 >> Host: consul-service:8500
2022-08-29 10:15:05.572 [configWatchTaskScheduler-1] DEBUG org.apache.http.headers - http-outgoing-0 >> Connection: Keep-Alive
2022-08-29 10:15:05.572 [configWatchTaskScheduler-1] DEBUG org.apache.http.headers - http-outgoing-0 >> User-Agent: Apache-HttpClient/4.5.13 (Java/1.8.0_101)
2022-08-29 10:15:05.573 [configWatchTaskScheduler-1] DEBUG org.apache.http.headers - http-outgoing-0 >> Accept-Encoding: gzip,deflate
2022-08-29 10:15:05.583 [configWatchTaskScheduler-1] DEBUG org.apache.http.headers - http-outgoing-0 << HTTP/1.1 404 Not Found
2022-08-29 10:15:05.583 [configWatchTaskScheduler-1] DEBUG org.apache.http.headers - http-outgoing-0 << Vary: Accept-Encoding
2022-08-29 10:15:05.583 [configWatchTaskScheduler-1] DEBUG org.apache.http.headers - http-outgoing-0 << X-Consul-Index: 675782
2022-08-29 10:15:05.583 [configWatchTaskScheduler-1] DEBUG org.apache.http.headers - http-outgoing-0 << X-Consul-Knownleader: true
2022-08-29 10:15:05.583 [configWatchTaskScheduler-1] DEBUG org.apache.http.headers - http-outgoing-0 << X-Consul-Lastcontact: 0
2022-08-29 10:15:05.583 [configWatchTaskScheduler-1] DEBUG org.apache.http.headers - http-outgoing-0 << Date: Mon, 29 Aug 2022 02:15:13 GMT
2022-08-29 10:15:05.583 [configWatchTaskScheduler-1] DEBUG org.apache.http.headers - http-outgoing-0 << Content-Length: 0
2022-08-29 10:15:05.583 [configWatchTaskScheduler-1] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection can be kept alive indefinitely
2022-08-29 10:15:05.583 [configWatchTaskScheduler-1] DEBUG o.a.h.i.conn.PoolingHttpClientConnectionManager - Connection [id: 0][route: {}->http://consul-service:8500] can be kept alive indefinitely
2022-08-29 10:15:05.583 [configWatchTaskScheduler-1] DEBUG o.a.h.i.conn.DefaultManagedHttpClientConnection - http-outgoing-0: set socket timeout to 0
2022-08-29 10:15:05.583 [configWatchTaskScheduler-1] DEBUG o.a.h.i.conn.PoolingHttpClientConnectionManager - Connection released: [id: 0][route: {}->http://consul-service:8500][total available: 1; route allocated: 2 of 500; total allocated: 2 of 1000]

HttpClient 线程池

HttpComponents HttpClient连接池(1)-结构
https://cloud.tencent.com/developer/article/1607662

HttpComponents HttpClient连接池(2)-连接的申请
https://cloud.tencent.com/developer/article/1607744

HttpClient中每个host+port有独立的连接池

httpclient 连接池对于每一个 host + port 定义为唯一 route, 参考 DefaultHttpRoutePlanner.determineRoute() 方法。
对于每一个 route 都有一个 RouteSpecificPool 类型对象对应。这个对象也是一个连接池,既在 httpclient 连接池里,对每一个 route 访问都独立建立各自的连接池,从而实现不同 route 访问连接池隔离。在连接池对象 Cpool 里由 Map 存储,key 为 route,value 为RouteSpecificPool 对象。

http 连接的申请主要调用 Cpool 对象(在父类 AbstractConnPool )的 lease() 方法

AbstractHttpClient.doExecute() 调用
connManager.requestConnection() 调用
PoolingClientConnectionManager.requestConnection() 调用
this.pool.lease(route, state) 返回一个 Future 调用起 get() 里面会调用 getPoolEntryBlocking() 方法,其中调用
getPool(route) 获取连接池

final RouteSpecificPool<T, C, E> pool = getPool(route);

所以每个 route 间的线程池是隔离的。


日志中不断重复的报warn:
00:00:00.665 [http-nio-8380-exec-454] WARN o.a.h.c.p.ResponseProcessCookies - Cookie rejected [BAIDUID=”C303600CB4B1BEDDCC9551003BD54DD5:FG=1”, version:0, domain:baidu.com, path:/, expiry:Wed Jan 12 00:00:46 CST 2022] Illegal ‘domain’ attribute “baidu.com”. Domain of origin: “10.123.123.21”

原因:服务器端 cookie 的 domain 是 10.123.123.21, 但客户端给传了个 baidu.com,两者不一致导致Cookie rejected。
domain是用来在浏览器端限制哪个域名可以访问此cookie内容的。我们使用httpclient一般来说是不需要用到cookie的。
为了不让日志不停的打印,可以设置HttpClient的Cookie策略:

RequestConfig requestConfig = = RequestConfig.custom()
                    // 设置Cookie策略为忽略
                    .setCookieSpec(CookieSpecs.STANDARD_STRICT)
                    .build();

从HTTPResponse中读取Json

利用 BufferedReader.readLine() 从 HTTPResponse.getEntity().getContent() 流中逐行读取返回内容:

HttpResponse response; // some response object
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
StringBuilder builder = new StringBuilder();
for (String line = null; (line = reader.readLine()) != null;) {
    builder.append(line).append("\n");
}
JSONTokener tokener = new JSONTokener(builder.toString());
JSONArray finalResult = new JSONArray(tokener);

如果 JSON 实际上是一行,那么你也可以删除循环和 builder:

HttpResponse response; // some response object
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
String json = reader.readLine();
JSONTokener tokener = new JSONTokener(json);
JSONArray finalResult = new JSONArray(tokener);

使用gson:

void getJson() throws IOException {
    HttpClient  httpClient = new DefaultHttpClient();
    HttpGet httpGet = new HttpGet("some url of json");
    HttpResponse httpResponse = httpClient.execute(httpGet);
    String response = EntityUtils.toString(httpResponse.getEntity());

    Gson gson = new Gson();
    MyClass myClassObj = gson.fromJson(response, MyClass.class);
}

使用实例:

HttpResponse httpResponse = UdsHttpClient.postForRawResponseWithHeader(server + QUERY_SO_ORDER_LIST_API, null, content, false, headersMap, logger);
checkState(httpResponse != null, "Artemis returned empty response");
checkState(httpResponse.getStatusLine() != null, "Artemis return error response");
checkState(httpResponse.getEntity() != null, "Artemis returned empty response entity");
checkState(httpResponse.getStatusLine().getStatusCode() != ARTEMIS_TOKEN_ERROR_CODE, "Artemis return token error code");
BufferedReader reader = new BufferedReader(new InputStreamReader(httpResponse.getEntity().getContent(), "UTF-8"));
String result = reader.readLine();
QuerySOListResponse response = JSONUtils.readValue(result, QuerySOListResponse.class);

How do I parse JSON from a Java HTTPResponse?
https://stackoverflow.com/questions/2845599/how-do-i-parse-json-from-a-java-httpresponse

如何从Java HTTPResponse解析JSON?
https://codeday.me/bug/20170629/34297.html


上一篇 Java-枚举

下一篇 Apache-Cassandra

阅读
评论
1.4k
阅读预计7分钟
创建日期 2018-08-24
修改日期 2022-08-29
类别
标签

页面信息

location:
protocol:
host:
hostname:
origin:
pathname:
href:
document:
referrer:
navigator:
platform:
userAgent:

评论