锘??xml version="1.0" encoding="utf-8" standalone="yes"?>北京赛车pk10加减规律:BlogJava - ▼幸运赛车规则▼幸运赛车投注▼幸运赛车开奖结果查询▼湖南幸运赛车开奖奖金▼幸运赛车走势▼//www.cqk4s.cn/DLevin/In general the OO style is to use a lot of little objects with a lot of little methods that give us a lot of plug points for overriding and variation. To do is to be -Nietzsche, To bei is to do -Kant, Do be do be do -Sinatrazh-cnSun, 24 Mar 2019 20:15:48 GMTSun, 24 Mar 2019 20:15:48 GMT60浣跨敤NamedParameterJdbcTemplate閬囧埌鏃犳硶浣跨敤鐨勫潙 - ▼幸运赛车规则▼幸运赛车投注▼幸运赛车开奖结果查询▼湖南幸运赛车开奖奖金▼幸运赛车走势▼//www.cqk4s.cn/DLevin/archive/2015/11/11/428149.htmlDLevinDLevinWed, 11 Nov 2015 10:46:00 GMT//www.cqk4s.cn/DLevin/archive/2015/11/11/428149.html//www.cqk4s.cn/DLevin/comments/428149.html//www.cqk4s.cn/DLevin/archive/2015/11/11/428149.html#Feedback0//www.cqk4s.cn/DLevin/comments/commentRss/428149.html//www.cqk4s.cn/DLevin/services/trackbacks/428149.html鏈杩戜竴鐩村湪鎹i紦HBase鐨勯」鐩紝涔嬪墠鍐欎簡涓浜涗唬鐮佷粠鏁版嵁搴撳姞杞芥暟鎹埌HBase锛屾墍鏈夌殑浠g爜閮借窇寰楀ソ濂藉湴锛岀劧鑰屼粖澶╁皾璇曠潃鎹簡涓涓暟鎹簱锛屽氨璺戜笉閫氫簡銆傞氳繃鏁版嵁宸ュ叿锛屽彲浠ュ彂鐜拌繛鎺ユ病鏈夐棶棰橈紝鑰屼笖鏈夐儴鍒嗛昏緫寰堥『鍒╅氳繃浜嗭紝鐒惰屾湁涓浜涘氨鏄崱涓讳簡锛岄氳繃jstack鎵撳嵃鍑烘潵鐨勪俊鎭彲浠ユ壘鍒拌繖鏍风殑鍫嗘爤锛?/span>
"runner{object-loader#292}-objecthandler" #323 prio=5 os_prio=0 tid=0x00002aaadc5ec800 nid=0x7f62 in Object.wait() [0x0000000056ce4000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1104)
        - locked <0x00000007736013e8> (a org.apache.commons.pool.impl.GenericObjectPool$Latch)
        at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
        at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
        at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
        at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)
鎵浠ュ紑濮嬫垜鎬鐤戞槸杩炴帴鐨勯棶棰橈紝浠庣綉涓婁篃鎵惧埌浜嗕竴涓被鍨嬬殑鐜拌薄锛屾湁浜烘鐤戞槸DBPC鐨勪竴涓猙ug瀵艰嚧姝婚攣锛//stackoverflow.com/questions/5714511/deadlock-issue-in-dbcp-deployed-on-tomcat锛屾墍浠ユ垜鍗囩骇浜咲BCP鐗堟湰1.4锛岀劧鑰屽拰杩欎汉涓鏍风殑缁撴灉锛屽崌绾BCP鐗堟湰骞舵病鏈夎В鍐抽棶棰樸傜畝鍗曠殑鐪婦BCP鐨勪唬鐮侊紝閮藉紑濮嬫鐤戞槸涓嶆槸鍥犱负娌℃湁Spring JdbcTemplate娌℃湁姝g‘鐨勬妸Connection杩斿洖鍥炲幓寮曡捣娉勬紡浜嗭紝鐒惰屼篃鏈夌偣鎰熻涓嶅お鍙兘锛屽洜涓鸿繖娈典唬鐮佸湪鍏朵粬鏁版嵁搴撻兘璺戝緱濂藉ソ鍦帮紝浣嗘槸鎴戜滑鐨勬暟鎹簱鐗堟湰閮芥槸涓鑷寸殑锛岀劧鑰屽叾浠栭厤缃笂涔熻鍋囪涓鑷翠簡锛堣蹇界暐鐨勪竴涓噸瑕佺殑鐐癸級銆?br />
鍚庢潵寮濮嬭皟閰嶇疆锛屽噺灏戣繛鎺ユ暟锛屽噺灏戠嚎绋嬫暟锛岀粡杩囧悇绉嶇粍鍚堬紝鍙戠幇褰撴妸DB璇荤殑batch闄嶅埌1鐨勬椂鍊欏氨鍙互work浜嗭紝闈炲父璇″紓鐨勪竴涓棶棰樸備粠鏁版嵁宸ュ叿涓煡鍒帮紝濡傛灉鐢╞atch锛屽緱鍒扮殑SQL鏄?
SELECT <column>, <column> FROM <table> where iid in (@p0, @p1)
濡傛灉鏄痓atch鏄?鐨勮瘽锛?br />
SELECT <column>, <column> FROM <table> where iid in (@p0)
杩欐SQL璇彞鏄繖涔堜骇鐢熺殑锛?br />
DataSource dataSource = ....
this.jdbc = new NamedParameterJdbcTemplate(dataSource);
...
MapSqlParameterSource parameters = new MapSqlParameterSource();
parameters.addValue("params", paramsMap.keySet());
jdbc.query("SELECT <columns> FROM <table> where <column> in (:params)";, parameters, new ResultSetExtractor<Void>() {
....
})

濡傛灉鏄竴涓猙atch鐨勮瘽锛屽湪jstack鍫嗘爤涓彲浠ョ湅鍒板畠涓鐩村湪绛夋暟鎹簱鐨勮繑鍥炵粨鏋滐細
"runner{object-loader#16}-objecthandler" #47 prio=5 os_prio=0 tid=0x0000000006ddd800 nid=0x2694 runnable [0x0000000045434000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:170)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at com.sybase.jdbc3.timedio.RawDbio.reallyRead(Unknown Source)
        at com.sybase.jdbc3.timedio.Dbio.doRead(Unknown Source)
        at com.sybase.jdbc3.timedio.InStreamMgr.a(Unknown Source)
        at com.sybase.jdbc3.timedio.InStreamMgr.doRead(Unknown Source)
        at com.sybase.jdbc3.tds.TdsProtocolContext.getChunk(Unknown Source)
杩欎篃瑙i噴浜嗙涓涓爢鏍堜竴鐩村仠鍦╞orrowObject(getConnection)鐨勯樁娈碉紝鍥犱负涔嬪墠鎵鏈夌殑Connection閮藉湪鏁版嵁搴撳牭浣忔病鏈夎繑鍥烇紝鎵浠ヨ繖涓嚎绋嬪啀鎷緾onnection鐨勬椂鍊欒秴杩囦簡鎴戣缃殑鏈澶onnection鏁帮紝鎵浠ュ氨绛夌潃鎷夸笉鍒癈onnection銆?br />
鍦ㄥ悗鏉ユ煡浜嗕竴涓嬩笉鍚屾暟鎹簱鐨凧DBC Driver淇℃伅(sp_version):
jConnect (TM) for JDBC(TM)/7.07 ESD #4 (Build 26793)/P/EBF20302/JDK 1.6.0/jdbcmain/OPT/Thu Jul  5 22:08:44 PDT 2012
jConnect (TM) for JDBC(TM)/1000/Wed Mar 11 05:01:24 2015 PDT

涔熷氨鏄杩欑鐢ㄦ硶鏄洜涓烘棫鐨凧DBC Driver瀵筃amedParameterJdbcTemplate涓嶅畬鍠勫紩璧风殑锛岃繖涓潙鑺变簡鎴戜竴鏁村ぉ鐨勬椂闂淬傘傘傘?img src ="//www.cqk4s.cn/DLevin/aggbug/428149.html" width = "1" height = "1" />

DLevin 2015-11-11 18:46 鍙戣〃璇勮
]]>
SSTable璇﹁В - ▼幸运赛车规则▼幸运赛车投注▼幸运赛车开奖结果查询▼湖南幸运赛车开奖奖金▼幸运赛车走势▼//www.cqk4s.cn/DLevin/archive/2015/09/25/427481.htmlDLevinDLevinThu, 24 Sep 2015 17:35:00 GMT//www.cqk4s.cn/DLevin/archive/2015/09/25/427481.html//www.cqk4s.cn/DLevin/comments/427481.html//www.cqk4s.cn/DLevin/archive/2015/09/25/427481.html#Feedback0//www.cqk4s.cn/DLevin/comments/commentRss/427481.html//www.cqk4s.cn/DLevin/services/trackbacks/427481.html 鍓嶈鍑犲勾鍓嶅湪璇籊oogle鐨凚igTable璁烘枃鐨勬椂鍊欙紝褰撴椂骞舵病鏈夌悊瑙h鏂囬噷闈㈣〃杈剧殑鎬濇兂锛屽洜鑰屽洬鍥靛悶鏋o紝骞舵病鏈夋敞鎰忓埌SSTable鐨勬蹇点傚啀鍚庢潵寮濮嬪叧娉℉Base鐨勮璁″拰婧愮爜鍚庯紝寮濮嬪BigTable浼犻掔殑鎬濇兂鎱㈡參鐨勬竻鏅拌捣鏉ワ紝浣嗘槸鍥犱负浜嬫儏澶锛屾病鏈夊畨鎺掑嚭鏃堕棿閲嶈BigTable鐨勮鏂囥傚湪椤圭洰閲岋紝鎴戝洜涓鸿嚜宸卞湪瀛Base锛屽紑濮嬩富鎺℉Base锛岃屽彟涓涓悓浜嬪垯鍥犱负瀵笴assandra姣旇緝鎰熷啋锛屽洜鑰屼粬涓昏鍏虫敞Cassandra鐨勮璁★紝涓嶈繃鎴戜滑涓や釜浜哄伓灏旈兘浼氳璁轰竴涓嬫妧鏈佽璁$殑鍚勭瑙傜偣鍜屽績寰楋紝鐒跺悗浠栧伓鐒剁殑璇翠簡涓鍙ワ細Cassandra鍜孒Base閮介噰鐢⊿STable鏍煎紡瀛樺偍锛岀劧鍚庢垜鏈兘鐨勯棶浜嗕竴鍙ワ細浠涔堟槸SSTable锛熶粬骞舵病鏈夊洖绛旓紝鍙兘涔熶笉鏄偅涔堝嚑鍙ヨ兘璇存竻妤氱殑锛屾垨鑰呬粬鑷繁涔熸病鏈夊皾璇曠殑鍘婚棶杩囪嚜宸辫繖涓棶棰樸傜劧鑰岃繖涓棶棰樻湰韬嵈涓鐩村洶鎵扮潃鎴戯紝鍥犺岃秮鐫鐜板湪鏈変竴浜涙椂闂存繁鍏ュ涔燞Base鍜孋assandra鐩稿叧璁捐鐨勬椂鍊欏厛鎶婅繖涓棶棰樺紕娓呮浜嗐?br />

SSTable鐨勫畾涔?/h2>瑕佽В閲婅繖涓湳璇殑鐪熸鍚箟锛屾渶濂界殑鏂规硶灏辨槸浠庡畠鐨勫嚭澶勬壘绛旀锛屾墍浠ラ噸鏂扮炕寮BigTable鐨勮鏂囥傚湪杩欑瘒璁烘枃涓紝鏈鍒濆SSTable鏄繖涔堟弿杩扮殑锛堢涓夐〉鏈拰绗洓椤靛垵锛夛細
SSTable

The Google SSTable file format is used internally to store Bigtable data. An SSTable provides a persistent, ordered immutable map from keys to values, where both keys and values are arbitrary byte strings. Operations are provided to look up the value associated with a specified key, and to iterate over all key/value pairs in a specified key range. Internally, each SSTable contains a sequence of blocks (typically each block is 64KB in size, but this is configurable). A block index (stored at the end of the SSTable) is used to locate blocks; the index is loaded into memory when the SSTable is opened. A lookup can be performed with a single disk seek: we first find the appropriate block by performing a binary search in the in-memory index, and then reading the appropriate block from disk. Optionally, an SSTable can be completely mapped into memory, which allows us to perform lookups and scans without touching disk.

绠鍗曠殑闈炵洿璇戯細
SSTable鏄疊igtable鍐呴儴鐢ㄤ簬鏁版嵁鐨勬枃浠舵牸寮忥紝瀹冪殑鏍煎紡涓烘枃浠舵湰韬氨鏄竴涓帓搴忕殑銆佷笉鍙彉鐨勩佹寔涔呯殑Key/Value瀵筂ap锛屽叾涓璌ey鍜寁alue閮藉彲浠ユ槸浠绘剰鐨刡yte瀛楃涓层備娇鐢↘ey鏉ユ煡鎵綱alue锛屾垨閫氳繃缁欏畾Key鑼冨洿閬嶅巻鎵鏈夌殑Key/Value瀵广傛瘡涓猄STable鍖呭惈涓绯诲垪鐨凚lock锛堜竴鑸珺lock澶у皬涓?4KB锛屼絾鏄畠鏄彲閰嶇疆鐨勶級锛屽湪SSTable鐨勬湯灏炬槸Block绱㈠紩锛岀敤浜庡畾浣岯lock锛岃繖浜涚储寮曞湪SSTable鎵撳紑鏃惰鍔犺浇鍒板唴瀛樹腑锛屽湪鏌ユ壘鏃堕鍏堜粠鍐呭瓨涓殑绱㈠紩浜屽垎鏌ユ壘鎵惧埌Block锛岀劧鍚庝竴娆$鐩樺閬撳嵆鍙鍙栧埌鐩稿簲鐨凚lock銆傝繕鏈変竴绉嶆柟妗堟槸灏嗚繖涓猄STable鍔犺浇鍒板唴瀛樹腑锛屼粠鑰屽湪鏌ユ壘鍜屾壂鎻忎腑涓嶉渶瑕佽鍙栫鐩樸?/span>

杩欎釜璨屼技灏辨槸HFile绗竴涓増鏈殑鏍煎紡涔堬紝璐村紶鍥炬劅鍙椾竴涓嬶細

鍦℉Base浣跨敤杩囩▼涓紝瀵硅繖涓増鏈殑HFile閬囧埌浠ヤ笅涓浜涢棶棰橈紙鍙傝?a >杩欓噷锛夛細
1. 瑙f瀽鏃跺唴瀛樹娇鐢ㄩ噺姣旇緝楂樸?br />2. Bloom Filter鍜孊lock绱㈠紩浼氬彉鐨勫緢澶э紝鑰屽奖鍝嶅惎鍔ㄦц兘銆傚叿浣撶殑锛孊loom Filter鍙互澧為暱鍒?00MB姣忎釜HFile锛岃孊lock绱㈠紩鍙互澧為暱鍒?00MB锛屽鏋滀竴涓狧RegionServer涓湁20涓狧Region锛屽垯浠栦滑鍒嗗埆鑳藉闀垮埌2GB鍜?GB鐨勫ぇ灏忋侶Region闇瑕佸湪鎵撳紑鏃讹紝闇瑕佸姞杞芥墍鏈夌殑Block绱㈠紩鍒板唴瀛樹腑锛屽洜鑰屽奖鍝嶅惎鍔ㄦц兘锛涜屽湪绗竴娆equest鏃讹紝闇瑕佸皢鏁翠釜Bloom Filter鍔犺浇鍒板唴瀛樹腑锛屽啀寮濮嬫煡鎵撅紝鍥犺孊loom Filter澶ぇ浼氬奖鍝嶇涓娆¤姹傜殑寤惰繜銆?br />鑰孒File鍦ㄧ増鏈?涓杩欎簺闂鍋氫簡涓浜涗紭鍖栵紝鍏蜂綋浼氬湪HFile瑙f瀽鏃惰缁嗚鏄庛?br />

SSTable浣滀负瀛樺偍浣跨敤

缁х画BigTable鐨勮鏂囧線涓嬭蛋锛屽湪5.3 Tablet Serving灏忚妭涓繖鏍峰啓閬擄紙绗?椤碉級锛?br />
Tablet Serving

Updates are committed to a commit log that stores redo records. Of these updates, the recently committed ones are stored in memory in a sorted buffer called a memtable; the older updates are stored in a sequence of SSTables. To recover a tablet, a tablet server reads its metadata from the METADATA table. This metadata contains the list of SSTables that comprise a tablet and a set of a redo points, which are pointers into any commit logs that may contain data for the tablet. The server reads the indices of the SSTables into memory and reconstructs the memtable by applying all of the updates that have committed since the redo points.

When a write operation arrives at a tablet server, the server checks that it is well-formed, and that the sender is authorized to perform the mutation. Authorization is performed by reading the list of permitted writers from a Chubby file (which is almost always a hit in the Chubby client cache). A valid mutation is written to the commit log. Group commit is used to improve the throughput of lots of small mutations [13, 16]. After the write has been committed, its contents are inserted into the memtable.

When a read operation arrives at a tablet server, it is similarly checked for well-formedness and proper authorization. A valid read operation is executed on a merged view of the sequence of SSTables and the memtable. Since the SSTables and the memtable are lexicographically sorted data structures, the merged view can be formed efficiently.

Incoming read and write operations can continue while tablets are split and merged.

绗竴娈靛拰绗笁娈电畝鍗曟弿杩帮紝闈炵炕璇戯細
鍦ㄦ柊鏁版嵁鍐欏叆鏃讹紝杩欎釜鎿嶄綔棣栧厛鎻愪氦鍒版棩蹇椾腑浣滀负redo绾綍锛屾渶杩戠殑鏁版嵁瀛樺偍鍦ㄥ唴瀛樼殑鎺掑簭缂撳瓨memtable涓紱鏃х殑鏁版嵁瀛樺偍鍦ㄤ竴绯诲垪鐨凷STable 涓傚湪recover涓紝tablet server浠嶮ETADATA琛ㄤ腑璇诲彇metadata锛宮etadata鍖呭惈浜嗙粍鎴怲ablet鐨勬墍鏈塖STable锛堢邯褰曚簡杩欎簺SSTable鐨勫厓 鏁版嵁淇℃伅锛屽SSTable鐨勪綅缃丼tartKey銆丒ndKey绛夛級浠ュ強涓绯诲垪鏃ュ織涓殑redo鐐广俆ablet Server璇诲彇SSTable鐨勭储寮曞埌鍐呭瓨锛屽苟replay杩欎簺redo鐐逛箣鍚庣殑鏇存柊鏉ラ噸鏋刴emtable銆?br />鍦ㄨ鏃讹紝瀹屾垚鏍煎紡銆佹巿鏉冪瓑妫鏌ュ悗锛岃浼氬悓鏃惰鍙朣STable銆乵emtable锛圚Base涓繕鍖呭惈浜咮lockCache涓殑鏁版嵁锛夊苟鍚堝苟浠栦滑鐨勭粨鏋滐紝鐢变簬SSTable鍜宮emtable閮芥槸瀛楀吀搴忔帓鍒楋紝鍥犺屽悎骞舵搷浣滃彲浠ュ緢楂樻晥瀹屾垚銆?br />

SSTable鍦–ompaction杩囩▼涓殑浣跨敤

鍦˙igTable璁烘枃5.4 Compaction灏忚妭涓槸杩欐牱璇寸殑锛?br />
Compaction

As write operations execute, the size of the memtable increases. When the memtable size reaches a threshold, the memtable is frozen, a new memtable is created, and the frozen memtable is converted to an SSTable and written to GFS. This minor compaction process has two goals: it shrinks the memory usage of the tablet server, and it reduces the amount of data that has to be read from the commit log during recovery if this server dies. Incoming read and write operations can continue while compactions occur.

Every minor compaction creates a new SSTable. If this behavior continued unchecked, read operations might need to merge updates from an arbitrary number of SSTables. Instead, we bound the number of such files by periodically executing a merging compaction in the background. A merging compaction reads the contents of a few SSTables and the memtable, and writes out a new SSTable. The input SSTables and memtable can be discarded as soon as the compaction has finished.

A merging compaction that rewrites all SSTables into exactly one SSTable is called a major compaction. SSTables produced by non-major compactions can contain special deletion entries that suppress deleted data in older SSTables that are still live. A major compaction, on the other hand, produces an SSTable that contains no deletion information or deleted data. Bigtable cycles through all of its tablets and regularly applies major compactions to them. These major compactions allow Bigtable to reclaim resources used by deleted data, and also allow it to ensure that deleted data disappears from the system in a timely fashion, which is important for services that store sensitive data.

闅忕潃memtable澶у皬澧炲姞鍒颁竴涓榾鍊硷紝杩欎釜memtable浼氳鍐讳綇鑰屽垱寤轰竴涓柊鐨刴emtable浠ヤ緵浣跨敤锛岃屾棫鐨刴emtable浼氳浆鎹㈡垚涓涓猄STable鑰屽啓閬揋FS涓紝杩欎釜杩囩▼鍙仛minor compaction銆傝繖涓猰inor compaction鍙互鍑忓皯鍐呭瓨浣跨敤閲忥紝骞跺彲浠ュ噺灏戞棩蹇楀ぇ灏忥紝鍥犱负鎸佷箙鍖栧悗鐨勬暟鎹彲浠ヤ粠鏃ュ織涓垹闄ゃ?/span>鍦╩inor compaction杩囩▼涓紝鍙互缁х画澶勭悊璇诲啓璇锋眰銆?br />姣忔minor compaction浼氱敓鎴愭柊鐨凷STable鏂囦欢锛屽鏋淪STable鏂囦欢鏁伴噺澧炲姞锛屽垯浼氬奖鍝嶈鐨勬ц兘锛屽洜鑰屾瘡娆¤閮介渶瑕佽鍙栨墍鏈塖STable鏂囦欢锛岀劧鍚庡悎骞剁粨鏋滐紝鍥犺屽SSTable鏂囦欢涓暟闇瑕佹湁涓婇檺锛屽苟涓旀椂涓嶆椂鐨勯渶瑕佸湪鍚庡彴鍋歮erging compaction锛岃繖涓猰erging compaction璇诲彇涓浜汼STable鏂囦欢鍜宮emtable鐨勫唴瀹癸紝骞跺皢浠栦滑鍚堝苟鍐欏叆涓涓柊鐨凷STable涓傚綋杩欎釜杩囩▼瀹屾垚鍚庯紝杩欎簺婧怱STable鍜宮emtable灏卞彲浠ヨ鍒犻櫎浜嗐?br />濡傛灉涓涓猰erging compaction鏄悎骞舵墍鏈塖STable鍒颁竴涓猄STable锛屽垯杩欎釜杩囩▼绉板仛major compaction銆備竴娆ajor compaction浼氬皢mark鎴愬垹闄ょ殑淇℃伅銆佹暟鎹垹闄わ紝鑰屽叾浠栦袱娆ompaction鍒欎細淇濈暀杩欎簺淇℃伅銆佹暟鎹紙mark鐨勫舰寮忥級銆侭igtable浼氭椂涓嶆椂鐨勬壂鎻忔墍鏈夌殑Tablet锛屽苟瀵瑰畠浠仛major compaction銆傝繖涓猰ajor compaction鍙互灏嗛渶瑕佸垹闄ょ殑鏁版嵁鐪熸鐨勫垹闄や粠鑰岃妭鐪佺┖闂达紝骞朵繚鎸佺郴缁熶竴鑷存с?/span>

SSTable鐨刲ocality鍜孖n Memory

鍦˙igtable涓紝瀹冪殑鏈湴鎬ф槸鐢盠ocality group鏉ュ畾涔夌殑锛屽嵆澶氫釜column family鍙互缁勫悎鍒颁竴涓猯ocality group涓紝鍦ㄥ悓涓涓猅ablet涓紝浣跨敤鍗曠嫭鐨凷STable瀛樺偍杩欎簺鍦ㄥ悓涓涓猯ocality group鐨刢olumn family銆侶Base鎶婅繖涓ā鍨嬬畝鍖栦簡锛屽嵆姣忎釜column family鍦ㄦ瘡涓狧Region閮戒娇鐢ㄥ崟鐙殑HFile瀛樺偍锛孒File娌℃湁locality group鐨勬蹇碉紝鎴栬呬竴涓猚olumn family灏辨槸涓涓猯ocality group銆?/span>

鍦˙igtable涓紝杩樺彲浠ユ敮鎸佸湪locality group绾у埆璁剧疆鏄惁灏嗘墍鏈夎繖涓猯ocality group鐨勬暟鎹姞杞藉埌鍐呭瓨涓紝鍦℉Base涓氳繃column family瀹氫箟鏃惰缃傝繖涓唴瀛樺姞杞介噰鐢ㄥ欢鏃跺姞杞斤紝涓昏搴旂敤浜庝竴浜涘皬鐨刢olumn family锛屽苟涓旂粡甯歌鐢ㄥ埌鐨勶紝浠庤屾彁鍗囪鐨勬ц兘锛屽洜鑰岃繖鏍峰氨涓嶉渶瑕佸啀浠庣鐩樹腑璇诲彇浜嗐?/span>

SSTable鍘嬬缉

Bigtable鐨勫帇缂╂槸鍩轰簬locality group绾у埆锛?br />
Compression

Clients can control whether or not the SSTables for a locality group are compressed, and if so, which compression format is used. The user-specified compression format is applied to each SSTable block (whose size is controllable via a locality group specific tuning parameter). Although we lose some space by compressing each block separately, we benefit in that small portions of an SSTable can be read without decompressing the entire file. Many clients use a two-pass custom compression scheme. The first pass uses Bentley and McIlroy’s scheme [6], which compresses long common strings across a large window. The second pass uses a fast compression algorithm that looks for repetitions in a small 16 KB window of the data. Both compression passes are very fast—they encode at 100–200 MB/s, and decode at 400–1000 MB/s on modern machines.

Bigtable鐨勫帇缂╀互SSTable涓殑涓涓狟lock涓哄崟浣嶏紝铏界劧姣忎釜Block涓哄帇缂╁崟浣嶆崯澶变竴浜涚┖闂达紝浣嗘槸閲囩敤杩欑鏂瑰紡锛屾垜浠彲浠ヤ互Block涓哄崟浣嶈鍙栥佽В鍘嬨佸垎鏋愶紝鑰屼笉鏄瘡娆′互涓涓?#8220;澶?#8221;鐨凷STable涓哄崟浣嶈鍙栥佽В鍘嬨佸垎鏋愩?/span>

SSTable鐨勮缂撳瓨

涓轰簡鎻愬崌璇荤殑鎬ц兘锛孊igtable閲囩敤涓ゅ眰缂撳瓨鏈哄埗锛?br />
Caching for read performance

To improve read performance, tablet servers use two levels of caching. The Scan Cache is a higher-level cache that caches the key-value pairs returned by the SSTable interface to the tablet server code. The Block Cache is a lower-level cache that caches SSTables blocks that were read from GFS. The Scan Cache is most useful for applications that tend to read the same data repeatedly. The Block Cache is useful for applications that tend to read data that is close to the data they recently read (e.g., sequential reads, or random reads of different columns in the same locality group within a hot row).

涓ゅ眰缂撳瓨鍒嗗埆鏄細
1. High Level锛岀紦瀛樹粠SSTable璇诲彇鐨凨ey/Value瀵广傛彁鍗囬偅浜涘惧悜閲嶅鐨勮鍙栫浉鍚岀殑鏁版嵁鐨勬搷浣滐紙寮曠敤灞閮ㄦу師鐞嗭級銆?br />2. Low Level锛孊lockCache锛岀紦瀛楽STable涓殑Block銆傛彁鍗囬偅浜涘惧悜浜庤鍙栫浉杩戞暟鎹殑鎿嶄綔銆?br />

Bloom Filter

鍓嶆枃鏈夋彁鍒癇igtable閲囩敤鍚堝苟璇伙紝鍗抽渶瑕佽鍙栨瘡涓猄STable涓殑鐩稿叧鏁版嵁锛屽苟鍚堝苟鎴愪竴涓粨鏋滆繑鍥烇紝鐒惰屾瘡娆¤閮介渶瑕佽鍙栨墍鏈塖STable锛岃嚜鐒朵細鑰楄垂鎬ц兘锛屽洜鑰屽紩鍏ヤ簡Bloom Filter锛屽畠鍙互寰堝揩閫熺殑鎵惧埌涓涓猂owKey涓嶅湪鏌愪釜SSTable涓殑浜嬪疄锛堟敞锛氬弽杩囨潵鍒欎笉鎴愮珛锛夈?br />
Bloom Filter

As described in Section 5.3, a read operation has to read from all SSTables that make up the state of a tablet. If these SSTables are not in memory, we may end up doing many disk accesses. We reduce the number of accesses by allowing clients to specify that Bloom fil- ters [7] should be created for SSTables in a particu- lar locality group. A Bloom filter allows us to ask whether an SSTable might contain any data for a spec- ified row/column pair. For certain applications, a small amount of tablet server memory used for storing Bloom filters drastically reduces the number of disk seeks re- quired for read operations. Our use of Bloom filters also implies that most lookups for non-existent rows or columns do not need to touch disk.

SSTable璁捐鎴怚mmutable鐨勫ソ澶?/h2>鍦⊿STable瀹氫箟涓氨鏈夋彁鍒癝STable鏄竴涓狪mmutable鐨刼rder map锛岃繖涓狪mmutable鐨勮璁″彲浠ヨ绯荤粺绠鍗曞緢澶氾細
Exploiting Immutability

Besides the SSTable caches, various other parts of the Bigtable system have been simplified by the fact that all of the SSTables that we generate are immutable. For example, we do not need any synchronization of accesses to the file system when reading from SSTables. As a result, concurrency control over rows can be implemented very efficiently. The only mutable data structure that is accessed by both reads and writes is the memtable. To reduce contention during reads of the memtable, we make each memtable row copy-on-write and allow reads and writes to proceed in parallel.

Since SSTables are immutable, the problem of permanently removing deleted data is transformed to garbage collecting obsolete SSTables. Each tablet’s SSTables are registered in the METADATA table. The master removes obsolete SSTables as a mark-and-sweep garbage collection [25] over the set of SSTables, where the METADATA table contains the set of roots.

Finally, the immutability of SSTables enables us to split tablets quickly. Instead of generating a new set of SSTables for each child tablet, we let the child tablets share the SSTables of the parent tablet.

鍏充簬Immutable鐨勪紭鐐规湁浠ヤ笅鍑犵偣锛?/span>
1. 鍦ㄨSSTable鏄笉闇瑕佸悓姝ャ傝鍐欏悓姝ュ彧闇瑕佸湪memtable涓鐞嗭紝涓轰簡鍑忓皯memtable鐨勮鍐欑珵浜夛紝Bigtable灏唌emtable鐨剅ow璁捐鎴恈opy-on-write锛屼粠鑰岃鍐欏彲浠ュ悓鏃惰繘琛屻?/span>
2. 姘镐箙鐨勭Щ闄ゆ暟鎹浆鍙樹负SSTable鐨凣arbage Collect銆傛瘡涓猅ablet涓殑SSTable鍦∕ETADATA琛ㄤ腑鏈夋敞鍐岋紝master浣跨敤mark-and-sweep绠楁硶灏哠STable鍦℅C杩囩▼涓Щ闄ゃ?/span>
3. 鍙互璁㏕ablet Split杩囩▼鍙樼殑楂樻晥锛屾垜浠笉闇瑕佷负姣忎釜瀛怲ablet鍒涘缓鏂扮殑SSTable锛岃屾槸鍙互鍏变韩鐖?/span>Tablet鐨凷STable銆?/span>

DLevin 2015-09-25 01:35 鍙戣〃璇勮
]]>[杞琞楂樻ц兘IO妯″瀷娴呮瀽 - ▼幸运赛车规则▼幸运赛车投注▼幸运赛车开奖结果查询▼湖南幸运赛车开奖奖金▼幸运赛车走势▼//www.cqk4s.cn/DLevin/archive/2015/09/04/427118.htmlDLevinDLevinFri, 04 Sep 2015 07:16:00 GMT//www.cqk4s.cn/DLevin/archive/2015/09/04/427118.html//www.cqk4s.cn/DLevin/comments/427118.html//www.cqk4s.cn/DLevin/archive/2015/09/04/427118.html#Feedback0//www.cqk4s.cn/DLevin/comments/commentRss/427118.html//www.cqk4s.cn/DLevin/services/trackbacks/427118.html北京赛车pk10加减规律:楂樻ц兘IO妯″瀷娴呮瀽

杞嚜锛//www.cnblogs.com/fanzhidongyzby/p/4098546.html

鏈嶅姟鍣ㄧ缂栫▼缁忓父闇瑕佹瀯閫犻珮鎬ц兘鐨処O妯″瀷锛屽父瑙佺殑IO妯″瀷鏈夊洓绉嶏細

锛?锛?/span>鍚屾闃诲IO锛圔locking IO锛夛細鍗充紶缁熺殑IO妯″瀷銆?/span>

锛?锛?/span>鍚屾闈為樆濉?/span>IO锛圢on-blocking IO锛夛細榛樿鍒涘缓鐨剆ocket閮芥槸闃诲鐨勶紝闈為樆濉濱O瑕佹眰socket琚缃负NONBLOCK銆傛敞鎰忚繖閲屾墍璇寸殑NIO骞堕潪Java鐨凬IO锛圢ew IO锛夊簱銆?/span>

锛?锛?/span>IO澶氳矾澶嶇敤锛圛O Multiplexing锛夛細鍗崇粡鍏哥殑Reactor璁捐妯″紡锛屾湁鏃朵篃绉颁负寮傛闃诲IO锛孞ava涓殑Selector鍜孡inux涓殑epoll閮芥槸杩欑妯″瀷銆?/span>

锛?锛?/span>寮傛IO锛圓synchronous IO锛夛細鍗崇粡鍏哥殑Proactor璁捐妯″紡锛屼篃绉颁负寮傛闈為樆濉濱O銆?/span>

鍚屾鍜屽紓姝?/span>鐨勬蹇垫弿杩扮殑鏄敤鎴风嚎绋嬩笌鍐呮牳鐨勪氦浜掓柟寮忥細鍚屾鏄寚鐢ㄦ埛绾跨▼鍙戣捣IO璇锋眰鍚庨渶瑕佺瓑寰呮垨鑰呰疆璇㈠唴鏍窱O鎿嶄綔瀹屾垚鍚庢墠鑳界户缁墽琛岋紱鑰屽紓姝ユ槸鎸囩敤鎴风嚎绋嬪彂璧稩O璇锋眰鍚庝粛缁х画鎵ц锛屽綋鍐呮牳IO鎿嶄綔瀹屾垚鍚庝細閫氱煡鐢ㄦ埛绾跨▼锛屾垨鑰呰皟鐢ㄧ敤鎴风嚎绋嬫敞鍐岀殑鍥炶皟鍑芥暟銆?/span>

闃诲鍜岄潪闃诲鐨勬蹇垫弿杩扮殑鏄敤鎴风嚎绋嬭皟鐢ㄥ唴鏍窱O鎿嶄綔鐨勬柟寮忥細闃诲鏄寚IO鎿嶄綔闇瑕佸交搴曞畬鎴愬悗鎵嶈繑鍥炲埌鐢ㄦ埛绌洪棿锛涜岄潪闃诲鏄寚IO鎿嶄綔琚皟鐢ㄥ悗绔嬪嵆杩斿洖缁欑敤鎴蜂竴涓姸鎬佸硷紝鏃犻渶绛夊埌IO鎿嶄綔褰诲簳瀹屾垚銆?/span>

鍙﹀锛?/span>Richard Stevens 鍦ㄣ奤nix 缃戠粶缂栫▼銆嬪嵎1涓彁鍒扮殑鍩轰簬淇″彿椹卞姩鐨処O锛圫ignal Driven IO锛夋ā鍨嬶紝鐢变簬璇ユā鍨嬪苟涓嶅父鐢紝鏈枃涓嶄綔娑夊強銆傛帴涓嬫潵锛屾垜浠缁嗗垎鏋愬洓绉嶅父瑙佺殑IO妯″瀷鐨勫疄鐜板師鐞嗐備负浜嗘柟渚挎弿杩帮紝鎴戜滑缁熶竴浣跨敤IO鐨勮鎿嶄綔浣滀负绀轰緥銆?/span>

涓銆?/span>鍚屾闃诲IO

鍚屾闃诲IO妯″瀷鏄渶绠鍗曠殑IO妯″瀷锛岀敤鎴风嚎绋嬪湪鍐呮牳杩涜IO鎿嶄綔鏃惰闃诲銆?/span>

鍥?/span>1 鍚屾闃诲IO

濡傚浘1鎵绀猴紝鐢ㄦ埛绾跨▼閫氳繃绯荤粺璋冪敤read鍙戣捣IO璇绘搷浣滐紝鐢辩敤鎴风┖闂磋浆鍒板唴鏍哥┖闂淬傚唴鏍哥瓑鍒版暟鎹寘鍒拌揪鍚庯紝鐒跺悗灏嗘帴鏀剁殑鏁版嵁鎷疯礉鍒扮敤鎴风┖闂达紝瀹屾垚read鎿嶄綔銆?/span>

鐢ㄦ埛绾跨▼浣跨敤鍚屾闃诲IO妯″瀷鐨勪吉浠g爜鎻忚堪涓猴細

{
    read(socket, buffer);
    process(buffer);
}

鍗崇敤鎴烽渶瑕佺瓑寰卹ead灏唖ocket涓殑鏁版嵁璇诲彇鍒癰uffer鍚庯紝鎵嶇户缁鐞嗘帴鏀剁殑鏁版嵁銆傛暣涓狪O璇锋眰鐨勮繃绋嬩腑锛岀敤鎴风嚎绋嬫槸琚樆濉炵殑锛岃繖瀵艰嚧鐢ㄦ埛鍦ㄥ彂璧稩O璇锋眰鏃讹紝涓嶈兘鍋氫换浣曚簨鎯咃紝瀵笴PU鐨勮祫婧愬埄鐢ㄧ巼涓嶅銆?/span>

浜屻?/span>鍚屾闈為樆濉濱O

鍚屾闈為樆濉濱O鏄湪鍚屾闃诲IO鐨勫熀纭涓婏紝灏唖ocket璁剧疆涓篘ONBLOCK銆傝繖鏍峰仛鐢ㄦ埛绾跨▼鍙互鍦ㄥ彂璧稩O璇锋眰鍚庡彲浠ョ珛鍗宠繑鍥炪?/span>

 

鍥? 鍚屾闈為樆濉濱O

濡傚浘2鎵绀猴紝鐢变簬socket鏄潪闃诲鐨勬柟寮忥紝鍥犳鐢ㄦ埛绾跨▼鍙戣捣IO璇锋眰鏃剁珛鍗宠繑鍥炪備絾骞舵湭璇诲彇鍒颁换浣曟暟鎹紝鐢ㄦ埛绾跨▼闇瑕佷笉鏂湴鍙戣捣IO璇锋眰锛岀洿鍒版暟鎹埌杈惧悗锛屾墠鐪熸璇诲彇鍒版暟鎹紝缁х画鎵ц銆?/span>

鐢ㄦ埛绾跨▼浣跨敤鍚屾闈為樆濉濱O妯″瀷鐨勪吉浠g爜鎻忚堪涓猴細

{
    
while(read(socket, buffer) != SUCCESS) { }
    process(buffer);
}

鍗? 鐢ㄦ埛闇瑕佷笉鏂湴璋冪敤read锛屽皾璇曡鍙杝ocket涓殑鏁版嵁锛岀洿鍒拌鍙栨垚鍔熷悗锛屾墠缁х画澶勭悊鎺ユ敹鐨勬暟鎹傛暣涓狪O璇锋眰鐨勮繃绋嬩腑锛岃櫧鐒剁敤鎴风嚎绋嬫瘡娆″彂璧稩O璇? 姹傚悗鍙互绔嬪嵆杩斿洖锛屼絾鏄负浜嗙瓑鍒版暟鎹紝浠嶉渶瑕佷笉鏂湴杞銆侀噸澶嶈姹傦紝娑堣椾簡澶ч噺鐨凜PU鐨勮祫婧愩備竴鑸緢灏戠洿鎺ヤ娇鐢ㄨ繖绉嶆ā鍨嬶紝鑰屾槸鍦ㄥ叾浠朓O妯″瀷涓娇鐢ㄩ潪闃? 濉濱O杩欎竴鐗规с?/span>

涓夈?/span>IO澶氳矾澶嶇敤

IO澶氳矾澶嶇敤妯″瀷鏄缓绔嬪湪鍐呮牳鎻愪緵鐨勫璺垎绂诲嚱鏁皊elect鍩虹涔嬩笂鐨勶紝浣跨敤select鍑芥暟鍙互閬垮厤鍚屾闈為樆濉濱O妯″瀷涓疆璇㈢瓑寰呯殑闂銆?/span>

鍥? 澶氳矾鍒嗙鍑芥暟select

濡傚浘3鎵绀猴紝鐢ㄦ埛棣栧厛灏嗛渶瑕佽繘琛孖O鎿嶄綔鐨剆ocket娣诲姞鍒皊elect涓紝鐒跺悗闃诲绛夊緟select绯荤粺璋冪敤杩斿洖銆傚綋鏁版嵁鍒拌揪鏃讹紝socket琚縺娲伙紝select鍑芥暟杩斿洖銆傜敤鎴风嚎绋嬫寮忓彂璧穜ead璇锋眰锛岃鍙栨暟鎹苟缁х画鎵ц銆?/span>

浠? 娴佺▼涓婃潵鐪嬶紝浣跨敤select鍑芥暟杩涜IO璇锋眰鍜屽悓姝ラ樆濉炴ā鍨嬫病鏈夊お澶х殑鍖哄埆锛岀敋鑷宠繕澶氫簡娣诲姞鐩戣socket锛屼互鍙婅皟鐢╯elect鍑芥暟鐨勯澶栨搷浣滐紝鏁? 鐜囨洿宸備絾鏄紝浣跨敤select浠ュ悗鏈澶х殑浼樺娍鏄敤鎴峰彲浠ュ湪涓涓嚎绋嬪唴鍚屾椂澶勭悊澶氫釜socket鐨処O璇锋眰銆傜敤鎴峰彲浠ユ敞鍐屽涓猻ocket锛岀劧鍚庝笉鏂湴璋? 鐢╯elect璇诲彇琚縺娲荤殑socket锛屽嵆鍙揪鍒板湪鍚屼竴涓嚎绋嬪唴鍚屾椂澶勭悊澶氫釜IO璇锋眰鐨勭洰鐨?/span>銆傝屽湪鍚屾闃诲妯″瀷涓紝蹇呴』閫氳繃澶氱嚎绋嬬殑鏂瑰紡鎵嶈兘杈惧埌杩欎釜鐩殑銆?/span>

鐢ㄦ埛绾跨▼浣跨敤select鍑芥暟鐨勪吉浠g爜鎻忚堪涓猴細

{
    select(socket);
    
while(1) {
        sockets 
= select();
        
for(socket in sockets) {
            
if(can_read(socket)) {
                read(socket, buffer);
                process(buffer);
            }
        }
    }
}

鍏朵腑while寰幆鍓嶅皢socket娣诲姞鍒皊elect鐩戣涓紝鐒跺悗鍦╳hile鍐呬竴鐩磋皟鐢╯elect鑾峰彇琚縺娲荤殑socket锛屼竴鏃ocket鍙锛屼究璋冪敤read鍑芥暟灏唖ocket涓殑鏁版嵁璇诲彇鍑烘潵銆?/span>

鐒? 鑰岋紝浣跨敤select鍑芥暟鐨勪紭鐐瑰苟涓嶄粎闄愪簬姝ゃ傝櫧鐒朵笂杩版柟寮忓厑璁稿崟绾跨▼鍐呭鐞嗗涓狪O璇锋眰锛屼絾鏄瘡涓狪O璇锋眰鐨勮繃绋嬭繕鏄樆濉炵殑锛堝湪select鍑芥暟涓婇樆 濉烇級锛屽钩鍧囨椂闂寸敋鑷虫瘮鍚屾闃诲IO妯″瀷杩樿闀裤傚鏋滅敤鎴风嚎绋嬪彧娉ㄥ唽鑷繁鎰熷叴瓒g殑socket鎴栬匢O璇锋眰锛岀劧鍚庡幓鍋氳嚜宸辩殑浜嬫儏锛岀瓑鍒版暟鎹埌鏉ユ椂鍐嶈繘琛屽 鐞嗭紝鍒欏彲浠ユ彁楂楥PU鐨勫埄鐢ㄧ巼銆?/span>

IO澶氳矾澶嶇敤妯″瀷浣跨敤浜哛eactor璁捐妯″紡瀹炵幇浜嗚繖涓鏈哄埗銆?/span>

鍥? Reactor璁捐妯″紡

濡? 鍥?鎵绀猴紝EventHandler鎶借薄绫昏〃绀篒O浜嬩欢澶勭悊鍣紝瀹冩嫢鏈塈O鏂囦欢鍙ユ焺Handle锛堥氳繃get_handle鑾峰彇锛夛紝浠ュ強瀵笻andle鐨? 鎿嶄綔handle_event锛堣/鍐欑瓑锛夈傜户鎵夸簬EventHandler鐨勫瓙绫诲彲浠ュ浜嬩欢澶勭悊鍣ㄧ殑琛屼负杩涜瀹氬埗銆俁eactor绫荤敤浜庣鐞? EventHandler锛堟敞鍐屻佸垹闄ょ瓑锛夛紝骞朵娇鐢╤andle_events瀹炵幇浜嬩欢寰幆锛屼笉鏂皟鐢ㄥ悓姝ヤ簨浠跺璺垎绂诲櫒锛堜竴鑸槸鍐呮牳锛夌殑澶氳矾鍒嗙鍑芥暟 select锛屽彧瑕佹煇涓枃浠跺彞鏌勮婵娲伙紙鍙/鍐欑瓑锛夛紝select灏辫繑鍥烇紙闃诲锛夛紝handle_events灏变細璋冪敤涓庢枃浠跺彞鏌勫叧鑱旂殑浜嬩欢澶勭悊鍣ㄧ殑 handle_event杩涜鐩稿叧鎿嶄綔銆?/span>

鍥?/span>5 IO澶氳矾澶嶇敤

濡? 鍥?鎵绀猴紝閫氳繃Reactor鐨勬柟寮忥紝鍙互灏嗙敤鎴风嚎绋嬭疆璇O鎿嶄綔鐘舵佺殑宸ヤ綔缁熶竴浜ょ粰handle_events浜嬩欢寰幆杩涜澶勭悊銆傜敤鎴风嚎绋嬫敞鍐屼簨浠跺鐞? 鍣ㄤ箣鍚庡彲浠ョ户缁墽琛屽仛鍏朵粬鐨勫伐浣滐紙寮傛锛夛紝鑰孯eactor绾跨▼璐熻矗璋冪敤鍐呮牳鐨剆elect鍑芥暟妫鏌ocket鐘舵併傚綋鏈塻ocket琚縺娲绘椂锛屽垯閫氱煡 鐩稿簲鐨勭敤鎴风嚎绋嬶紙鎴栨墽琛岀敤鎴风嚎绋嬬殑鍥炶皟鍑芥暟锛夛紝鎵цhandle_event杩涜鏁版嵁璇诲彇銆佸鐞嗙殑宸ヤ綔銆傜敱浜巗elect鍑芥暟鏄樆濉炵殑锛屽洜姝ゅ璺疘O澶嶇敤 妯″瀷涔熻绉颁负寮傛闃诲IO妯″瀷銆傛敞鎰忥紝杩欓噷鐨勬墍璇寸殑闃诲鏄寚select鍑芥暟鎵ц鏃剁嚎绋嬭闃诲锛岃屼笉鏄寚socket銆備竴鑸湪浣跨敤IO澶氳矾澶嶇敤妯″瀷 鏃讹紝socket閮芥槸璁剧疆涓篘ONBLOCK鐨勶紝涓嶈繃杩欏苟涓嶄細浜х敓褰卞搷锛屽洜涓虹敤鎴峰彂璧稩O璇锋眰鏃讹紝鏁版嵁宸茬粡鍒拌揪浜嗭紝鐢ㄦ埛绾跨▼涓瀹氫笉浼氳闃诲銆?/span>

鐢ㄦ埛绾跨▼浣跨敤IO澶氳矾澶嶇敤妯″瀷鐨勪吉浠g爜鎻忚堪涓猴細

void UserEventHandler::handle_event() {
    
if(can_read(socket)) {
        read(socket, buffer);
        process(buffer);
    }
}

{
    Reactor.register(
new UserEventHandler(socket));
}

鐢ㄦ埛闇瑕侀噸鍐橢ventHandler鐨刪andle_event鍑芥暟杩涜璇诲彇鏁版嵁銆佸鐞嗘暟鎹殑宸ヤ綔锛岀敤鎴风嚎绋嬪彧闇瑕佸皢鑷繁鐨凟ventHandler娉ㄥ唽鍒癛eactor鍗冲彲銆俁eactor涓環andle_events浜嬩欢寰幆鐨勪吉浠g爜澶ц嚧濡備笅銆?/span>

Reactor::handle_events() {
    
while(1) {
       sockets 
= select();
       
for(socket in sockets) {
            get_event_handler(socket).handle_event();
       }
    }
}

浜嬩欢寰幆涓嶆柇鍦拌皟鐢╯elect鑾峰彇琚縺娲荤殑socket锛岀劧鍚庢牴鎹幏鍙杝ocket瀵瑰簲鐨凟ventHandler锛屾墽琛屽櫒handle_event鍑芥暟鍗冲彲銆?/span>

IO澶氳矾澶嶇敤鏄渶甯镐娇鐢ㄧ殑IO妯″瀷锛屼絾鏄叾寮傛绋嬪害杩樹笉澶?#8220;褰诲簳”锛屽洜涓哄畠浣跨敤浜嗕細闃诲绾跨▼鐨剆elect绯荤粺璋冪敤銆傚洜姝O澶氳矾澶嶇敤鍙兘绉颁负寮傛闃诲IO锛岃岄潪鐪熸鐨勫紓姝O銆?/span>

鍥涖?/span>寮傛IO

“鐪? 姝?#8221;鐨勫紓姝O闇瑕佹搷浣滅郴缁熸洿寮虹殑鏀寔銆傚湪IO澶氳矾澶嶇敤妯″瀷涓紝浜嬩欢寰幆灏嗘枃浠跺彞鏌勭殑鐘舵佷簨浠堕氱煡缁欑敤鎴风嚎绋嬶紝鐢辩敤鎴风嚎绋嬭嚜琛岃鍙栨暟鎹佸鐞嗘暟鎹傝屽湪寮? 姝O妯″瀷涓紝褰撶敤鎴风嚎绋嬫敹鍒伴氱煡鏃讹紝鏁版嵁宸茬粡琚唴鏍歌鍙栧畬姣曪紝骞舵斁鍦ㄤ簡鐢ㄦ埛绾跨▼鎸囧畾鐨勭紦鍐插尯鍐咃紝鍐呮牳鍦↖O瀹屾垚鍚庨氱煡鐢ㄦ埛绾跨▼鐩存帴浣跨敤鍗冲彲銆?/span>

寮傛IO妯″瀷浣跨敤浜哖roactor璁捐妯″紡瀹炵幇浜嗚繖涓鏈哄埗銆?/span>

鍥? Proactor璁捐妯″紡

濡? 鍥?锛孭roactor妯″紡鍜孯eactor妯″紡鍦ㄧ粨鏋勪笂姣旇緝鐩镐技锛屼笉杩囧湪鐢ㄦ埛锛圕lient锛変娇鐢ㄦ柟寮忎笂宸埆杈冨ぇ銆俁eactor妯″紡涓紝鐢ㄦ埛绾跨▼閫氳繃 鍚慠eactor瀵硅薄娉ㄥ唽鎰熷叴瓒g殑浜嬩欢鐩戝惉锛岀劧鍚庝簨浠惰Е鍙戞椂璋冪敤浜嬩欢澶勭悊鍑芥暟銆傝孭roactor妯″紡涓紝鐢ㄦ埛绾跨▼灏? AsynchronousOperation锛堣/鍐欑瓑锛夈丳roactor浠ュ強鎿嶄綔瀹屾垚鏃剁殑CompletionHandler娉ㄥ唽鍒? AsynchronousOperationProcessor銆侫synchronousOperationProcessor浣跨敤Facade妯″紡鎻? 渚涗簡涓缁勫紓姝ユ搷浣淎PI锛堣/鍐欑瓑锛変緵鐢ㄦ埛浣跨敤锛屽綋鐢ㄦ埛绾跨▼璋冪敤寮傛API鍚庯紝渚跨户缁墽琛岃嚜宸辩殑浠诲姟銆? AsynchronousOperationProcessor 浼氬紑鍚嫭绔嬬殑鍐呮牳绾跨▼鎵ц寮傛鎿嶄綔锛屽疄鐜扮湡姝g殑寮傛銆傚綋寮傛IO鎿嶄綔瀹屾垚 鏃讹紝AsynchronousOperationProcessor灏嗙敤鎴风嚎绋嬩笌AsynchronousOperation涓璧锋敞鍐岀殑Proactor 鍜孋ompletionHandler鍙栧嚭锛岀劧鍚庡皢CompletionHandler涓嶪O鎿嶄綔鐨勭粨鏋滄暟鎹竴璧疯浆鍙戠粰 Proactor锛孭roactor璐熻矗鍥炶皟姣忎竴涓紓姝ユ搷浣滅殑浜嬩欢瀹屾垚澶勭悊鍑芥暟handle_event銆傝櫧鐒禤roactor妯″紡涓瘡涓紓姝ユ搷浣滈兘鍙互 缁戝畾涓涓狿roactor瀵硅薄锛屼絾鏄竴鑸湪鎿嶄綔绯荤粺涓紝Proactor琚疄鐜颁负Singleton妯″紡锛屼互渚夸簬闆嗕腑鍖栧垎鍙戞搷浣滃畬鎴愪簨浠躲?/span>

鍥?/span>7 寮傛IO

濡? 鍥?鎵绀猴紝寮傛IO妯″瀷涓紝鐢ㄦ埛绾跨▼鐩存帴浣跨敤鍐呮牳鎻愪緵鐨勫紓姝O API鍙戣捣read璇锋眰锛屼笖鍙戣捣鍚庣珛鍗宠繑鍥烇紝缁х画鎵ц鐢ㄦ埛绾跨▼浠g爜銆備笉杩囨鏃剁敤鎴风嚎绋嬪凡 缁忓皢璋冪敤鐨凙synchronousOperation鍜孋ompletionHandler娉ㄥ唽鍒板唴鏍革紝鐒跺悗鎿嶄綔绯荤粺寮鍚嫭绔嬬殑鍐呮牳绾跨▼鍘诲鐞咺O鎿? 浣溿傚綋read璇锋眰鐨勬暟鎹埌杈炬椂锛岀敱鍐呮牳璐熻矗璇诲彇socket涓殑鏁版嵁锛屽苟鍐欏叆鐢ㄦ埛鎸囧畾鐨勭紦鍐插尯涓傛渶鍚庡唴鏍稿皢read鐨勬暟鎹拰鐢ㄦ埛绾跨▼娉ㄥ唽鐨? CompletionHandler鍒嗗彂缁欏唴閮≒roactor锛孭roactor灏咺O瀹屾垚鐨勪俊鎭氱煡缁欑敤鎴风嚎绋嬶紙涓鑸氳繃璋冪敤鐢ㄦ埛绾跨▼娉ㄥ唽鐨勫畬鎴愪簨浠? 澶勭悊鍑芥暟锛夛紝瀹屾垚寮傛IO銆?/span>

鐢ㄦ埛绾跨▼浣跨敤寮傛IO妯″瀷鐨勪吉浠g爜鎻忚堪涓猴細


void UserCompletionHandler::handle_event(buffer) {
    process(buffer);
}

{
    aio_read(socket, 
new UserCompletionHandler);
}

鐢ㄦ埛闇瑕侀噸鍐機ompletionHandler鐨刪andle_event鍑芥暟杩涜澶勭悊鏁版嵁鐨勫伐浣滐紝鍙傛暟buffer琛ㄧずProactor宸茬粡鍑嗗濂界殑鏁版嵁锛岀敤鎴风嚎绋嬬洿鎺ヨ皟鐢ㄥ唴鏍告彁渚涚殑寮傛IO API锛屽苟灏嗛噸鍐欑殑CompletionHandler娉ㄥ唽鍗冲彲銆?/span>

鐩? 姣斾簬IO澶氳矾澶嶇敤妯″瀷锛屽紓姝O骞朵笉鍗佸垎甯哥敤锛屼笉灏戦珮鎬ц兘骞跺彂鏈嶅姟绋嬪簭浣跨敤IO澶氳矾澶嶇敤妯″瀷+澶氱嚎绋嬩换鍔″鐞嗙殑鏋舵瀯鍩烘湰鍙互婊¤冻闇姹傘傚喌涓旂洰鍓嶆搷浣滅郴缁熷 寮傛IO鐨勬敮鎸佸苟闈炵壒鍒畬鍠勶紝鏇村鐨勬槸閲囩敤IO澶氳矾澶嶇敤妯″瀷妯℃嫙寮傛IO鐨勬柟寮忥紙IO浜嬩欢瑙﹀彂鏃朵笉鐩存帴閫氱煡鐢ㄦ埛绾跨▼锛岃屾槸灏嗘暟鎹鍐欏畬姣曞悗鏀惧埌鐢ㄦ埛鎸囧畾鐨? 缂撳啿鍖轰腑锛夈侸ava7涔嬪悗宸茬粡鏀寔浜嗗紓姝O锛屾劅鍏磋叮鐨勮鑰呭彲浠ュ皾璇曚娇鐢ㄣ?/span>

鏈枃浠庡熀鏈蹇点佸伐浣滄祦绋嬪拰浠g爜绀? 渚嬩笁涓眰娆$畝瑕佹弿杩颁簡甯歌鐨勫洓绉嶉珮鎬ц兘IO妯″瀷鐨勭粨鏋勫拰鍘熺悊锛岀悊娓呬簡鍚屾銆佸紓姝ャ侀樆濉炪侀潪闃诲杩欎簺瀹规槗娣锋穯鐨勬蹇点傞氳繃瀵归珮鎬ц兘IO妯″瀷鐨勭悊瑙o紝鍙互鍦ㄦ湇 鍔$绋嬪簭鐨勫紑鍙戜腑閫夋嫨鏇寸鍚堝疄闄呬笟鍔$壒鐐圭殑IO妯″瀷锛屾彁楂樻湇鍔¤川閲忋傚笇鏈涙湰鏂囧浣犳湁鎵甯姪銆?/span>


鐩镐技鐨勶細
//www.cnblogs.com/nufangrensheng/p/3588690.html
//www.ibm.com/developerworks/cn/linux/l-async/



DLevin 2015-09-04 15:16 鍙戣〃璇勮
]]>Netty3鏋舵瀯瑙f瀽 - ▼幸运赛车规则▼幸运赛车投注▼幸运赛车开奖结果查询▼湖南幸运赛车开奖奖金▼幸运赛车走势▼//www.cqk4s.cn/DLevin/archive/2015/09/04/427031.htmlDLevinDLevinFri, 04 Sep 2015 01:40:00 GMT//www.cqk4s.cn/DLevin/archive/2015/09/04/427031.html//www.cqk4s.cn/DLevin/comments/427031.html//www.cqk4s.cn/DLevin/archive/2015/09/04/427031.html#Feedback0//www.cqk4s.cn/DLevin/comments/commentRss/427031.html//www.cqk4s.cn/DLevin/services/trackbacks/427031.html鍓嶈

寰堟棭浠ュ墠灏辨湁璇籒etty婧愮爜鐨勬墦绠椾簡锛岀劧鑰岀涓娆″皾璇曠殑鏃跺欎粠Netty4寮濮嬶紝涓鐩存姄涓嶅埌鏍稿績鐨勬鏋舵祦绋嬶紝鍚庢潵鍥犱负鍏朵粬浜嬫儏蹇欑潃灏辨斁涓嬩簡銆傝繖娆¤秮鐫浼戝亣閲嶆柊鎹¤捣杩欎釜纭澶达紝鍥犱负Netty3鐜板湪杩樺湪琚緢澶氶」鐩娇鐢紝鍥犺岃繖娆″喅瀹氬厛浠嶯etty3鍏ユ墜锛岀灛闂村彂鐜癗etty3鐨勪唬鐮佹瘮Netty4涓涓煩鐨勫锛屽緢澶氭蹇靛湪浠g爜鏈韩涓兘鏈夋竻鏅扮殑琛ㄨ揪锛屾墍浠ュ崐澶╁氨鎶婃暣涓鏋剁殑楠ㄦ灦鎼炴竻妤氫簡銆傚啀璇?a >Netty4瀵筃etty3鐨勬敼杩涙荤粨锛屽洖鍘昏Netty4鐨勬簮鐮侊紝鍙嶈岃寰楄交鏉句簡锛屼竴绉嶈眮鐒跺紑鏈楃殑鎰熻銆?br />
璁板緱鍘诲勾璇籎etty婧愮爜鐨勬椂鍊欙紝鍥犱负浠g爜澶簽澶э紝骞朵笖鑷繁鐨凥TTP Server鐨勪簡瑙eお灏戯紝鍥犺屽彧鑳借嚜搴曞悜涓婄殑涓涓竴涓ā鍧楃殑鍙犲姞锛岀洿鍒版渶鍚庢妸鎵浠ョ殑妯″潡杩炴帴鍦ㄤ竴璧疯岀湅娓呭畠鐨勭湡姝f牳蹇冮鏋躲傜幇鍦ㄨ婧愮爜锛屽紑濮嬩範鎯厛鎶婇鏋剁悊娓咃紝鐒跺悗寤朵几鍒颁笉鍚岀殑鍣ㄥ畼銆佽鑲夎岀湅娓呮暣涓汉浣撱?br />
鏈枃浠嶳eactor妯″紡鍦∟etty3涓殑搴旂敤锛屽紩鍑篘etty3鐨勬暣浣撴灦鏋勪互鍙婃帶鍒舵祦绋嬶紱鐒惰岄櫎浜哛eactor妯″紡锛孨etty3杩樺湪ChannelPipeline涓娇鐢ㄤ簡Intercepting Filter妯″紡锛岃繖涓ā寮忎篃鍦⊿ervlet鐨凢ilter涓垚鍔熶娇鐢紝鍥犺屾湰鏂囪繕浼氫粠Intercepting Filter妯″紡鍑哄彂璇︾粏浠嬬粛ChannelPipeline鐨勮璁$悊蹇点傛湰鏂囧亣璁捐鑰呭凡缁忓Netty鏈変竴瀹氱殑浜嗚В锛屽洜鑰屼笉浼氬寘鍚繃澶氬叆闂ㄤ粙缁嶏紝浠ュ強甯甆etty鍋氬浼犵殑鏂囧瓧銆?br />

Netty3涓殑Reactor妯″紡

Reactor妯″紡鍦∟etty涓簲鐢ㄩ潪甯告垚鍔燂紝鍥犺屽畠涔熸槸鍦∟etty涓彈澶ц倖瀹d紶鐨勬ā寮忥紝鍏充簬Reactor妯″紡鍙互璇︾粏鍙傝冩湰浜虹殑鍙︿竴绡囨枃绔?a href="//www.cqk4s.cn/DLevin/archive/2015/09/02/427045.html">銆奟eactor妯″紡璇﹁В銆?/a>锛屽Reactor妯″紡鐨勫疄鐜版槸Netty3鐨勫熀鏈鏋讹紝鍥犺屾湰灏忚妭浼氳缁嗕粙缁峈eactor妯″紡濡備綍搴旂敤Netty3涓?br />
濡傛灉璇汇奟eactor妯″紡璇﹁В銆嬶紝鎴戜滑鐭ラ亾Reactor妯″紡鐢盚andle銆丼ynchronous Event Demultiplexer銆両nitiation Dispatcher銆丒vent Handler銆丆oncrete Event Handler鏋勬垚锛屽湪Java鐨勫疄鐜扮増鏈腑锛孋hannel瀵瑰簲Handle锛孲elector瀵瑰簲Synchronous Event Demultiplexer锛屽苟涓擭etty3杩樹娇鐢ㄤ簡涓ゅ眰Reactor锛歁ain Reactor鐢ㄤ簬澶勭悊Client鐨勮繛鎺ヨ姹傦紝Sub Reactor鐢ㄤ簬澶勭悊鍜孋lient杩炴帴鍚庣殑璇诲啓璇锋眰锛堝叧浜庤繖涓蹇佃繕鍙互鍙傝僁oug Lea鐨勮繖绡嘝PT锛?a >Scalable IO In Java锛夈傛墍浠ユ垜浠厛瑕佽В鍐砃etty3涓娇鐢ㄤ粈涔堢被瀹炵幇鎵鏈夌殑涓婅堪妯″潡骞舵妸浠栦滑鑱旂郴鍦ㄤ竴璧风殑锛屼互NIO瀹炵幇鏂瑰紡涓轰緥锛?br />
妯″紡鏄竴绉嶆娊璞★紝浣嗘槸鍦ㄥ疄鐜颁腑锛岀粡甯镐細鍥犱负璇█鐗规с佹鏋跺拰鎬ц兘闇瑕佽屽仛涓浜涙敼鍙橈紝鍥犺孨etty3瀵筊eactor妯″紡鐨勫疄鐜版湁涓濂楄嚜宸辩殑璁捐锛?br />1. ChannelEvent锛?/strong>Reactor鏄熀浜庝簨浠剁紪绋嬬殑锛屽洜鑰屽湪Netty3涓娇鐢–hannelEvent鎶借薄鐨勮〃杈綨etty3鍐呴儴鍙互浜х敓鐨勫悇绉嶄簨浠讹紝鎵鏈夎繖浜涗簨浠跺璞″湪Channels甯姪绫讳腑浜х敓锛屽苟涓旂敱瀹冨皢浜嬩欢鎺ㄥ叆鍒癈hannelPipeline涓紝ChannelPipeline鏋勫缓ChannelHandler绠¢亾锛孋hannelEvent娴佺粡杩欎釜绠¢亾瀹炵幇鎵鏈夌殑涓氬姟閫昏緫澶勭悊銆侰hannelEvent瀵瑰簲鐨勪簨浠舵湁锛欳hannelStateEvent琛ㄧずChannel鐘舵佺殑鍙樺寲浜嬩欢锛岃屽鏋滃綋鍓岰hannel瀛樺湪Parent Channel锛屽垯璇ヤ簨浠惰繕浼氫紶閫掑埌Parent Channel鐨凜hannelPipeline涓紝濡侽PEN銆丅OUND銆丆ONNECTED銆両NTEREST_OPS绛夛紝璇ヤ簨浠跺彲浠ュ湪鍚勭涓嶅悓瀹炵幇鐨凜hannel銆丆hannelSink涓骇鐢燂紱MessageEvent琛ㄧず浠嶴ocket涓鍙栨暟鎹畬鎴愩侀渶瑕佸悜Socket鍐欐暟鎹垨ChannelHandler瀵瑰綋鍓峂essage瑙f瀽(濡侱ecoder銆丒ncoder)鍚庤Е鍙戠殑浜嬩欢锛屽畠鐢盢ioWorker銆侀渶瑕佸Message鍋氳繘涓姝ュ鐞嗙殑ChannelHandler浜х敓锛沇riteCompletionEvent琛ㄧず鍐欏畬鎴愯岃Е鍙戠殑浜嬩欢锛屽畠鐢盢ioWorker浜х敓锛汦xceptionEvent琛ㄧず鍦ㄥ鐞嗚繃绋嬩腑鍑虹幇鐨凟xception锛屽畠鍙互鍙戠敓鍦ㄥ悇涓瀯浠朵腑锛屽Channel銆丆hannelSink銆丯ioWorker銆丆hannelHandler涓紱IdleStateEvent鐢盜dleStateHandler瑙﹀彂锛岃繖涔熸槸涓涓狢hannelEvent鍙互鏃犵紳鎵╁睍鐨勪緥瀛愩傛敞锛氬湪Netty4鍚庯紝宸茬粡娌℃湁ChannelEvent绫伙紝鎵鏈変笉鍚屼簨浠堕兘鐢ㄥ搴旀柟娉曡〃杈撅紝杩欎篃鎰忓懗杩機hannelEvent涓嶅彲鎵╁睍锛孨etty4閲囩敤鍦–hannelInboundHandler涓姞鍏serEventTriggered()鏂规硶鏉ュ疄鐜拌繖绉嶆墿灞曪紝鍏蜂綋鍙互鍙傝?a >杩欓噷銆?br />2. ChannelHandler锛?/strong>鍦∟etty3涓紝ChannelHandler鐢ㄤ簬琛ㄧずReactor妯″紡涓殑EventHandler銆侰hannelHandler鍙槸涓涓爣璁版帴鍙o紝瀹冩湁涓や釜瀛愭帴鍙o細ChannelDownstreamHandler鍜孋hannelUpstreamHandler锛屽叾涓瑿hannelDownstreamHandler琛ㄧず浠庣敤鎴峰簲鐢ㄧ▼搴忔祦鍚慛etty3鍐呴儴鐩村埌鍚慡ocket鍐欐暟鎹殑绠¢亾锛屽湪Netty4涓敼鍚嶄负ChannelOutboundHandler锛汣hannelUpstreamHandler琛ㄧず鏁版嵁浠嶴ocket杩涘叆Netty3鍐呴儴鍚戠敤鎴峰簲鐢ㄧ▼搴忓仛鏁版嵁澶勭悊鐨勭閬擄紝鍦∟etty4涓敼鍚嶄负ChannelInboundHandler銆?br />3. ChannelPipeline锛?/strong>鐢ㄤ簬绠$悊ChannelHandler鐨勭閬擄紝姣忎釜Channel涓涓狢hannelPipeline瀹炰緥锛屽彲浠ヨ繍琛岃繃绋嬩腑鍔ㄦ佺殑鍚戣繖涓閬撲腑娣诲姞銆佸垹闄hannelHandler锛堢敱浜庡疄鐜扮殑闄愬埗锛屽湪鏈鏈鐨凜hannelHandler鍚戝悗娣诲姞鎴栧垹闄hannelHandler涓嶄竴瀹氬湪褰撳墠鎵ц娴佺▼涓捣鏁堬紝鍙傝?a >杩欓噷锛夈侰hannelPipeline鍐呴儴缁存姢涓涓狢hannelHandler鐨勫弻鍚戦摼琛紝瀹冧互Upstream(Inbound)鏂瑰悜涓烘鍚戯紝Downstream(Outbound)鏂瑰悜涓烘柟鍚戙侰hannelPipeline閲囩敤Intercepting Filter妯″紡瀹炵幇锛屽叿浣撳彲浠ュ弬鑰?a href="//www.cqk4s.cn/DLevin/archive/2015/09/03/427086.html">杩欓噷锛岃繖涓ā寮忕殑瀹炵幇鍦ㄥ悗涓鑺備腑杩樻槸璇︾粏浠嬬粛銆?br />4. NioSelector锛?/strong>Netty3浣跨敤NioSelector鏉ュ瓨鏀維elector锛圫ynchronous Event Demultiplexer锛夛紝姣忎釜鏂颁骇鐢熺殑NIO Channel閮藉悜杩欎釜Selector娉ㄥ唽鑷繁浠ヨ杩欎釜Selector鐩戝惉杩欎釜NIO Channel涓彂鐢熺殑浜嬩欢锛屽綋浜嬩欢鍙戠敓鏃讹紝璋冪敤甯姪绫籆hannels涓殑鏂规硶鐢熸垚ChannelEvent瀹炰緥锛屽皢璇ヤ簨浠跺彂閫佸埌杩欎釜Netty Channel瀵瑰簲鐨凜hannelPipeline涓紝鑰屼氦缁欏悇绾hannelHandler澶勭悊銆傚叾涓湪鍚慡elector娉ㄥ唽NIO Channel鏃讹紝Netty Channel瀹炰緥浠ttachment鐨勫舰寮忎紶鍏ワ紝璇etty Channel鍦ㄥ叾鍐呴儴鐨凬IO Channel浜嬩欢鍙戠敓鏃讹紝浼氫互Attachment鐨勫舰寮忓瓨鍦ㄤ簬SelectionKey涓紝鍥犺屾瘡涓簨浠跺彲浠ョ洿鎺ヤ粠杩欎釜Attachment涓幏鍙栫浉鍏抽摼鐨凬etty Channel锛屽苟浠嶯etty Channel涓幏鍙栦笌涔嬬浉鍏宠仈鐨凜hannelPipeline锛岃繖涓疄鐜板拰Doug Lea鐨?a >Scalable IO In Java涓妯′竴鏍枫傚彟澶朜etty3杩橀噰鐢ㄤ簡Scalable IO In Java涓浉鍚岀殑Main Reactor鍜孲ub Reactor璁捐锛屽叾涓璑ioSelector鐨勪袱涓疄鐜帮細Boss鍗充负Main Reactor锛孨ioWorker涓篠ub Reactor銆侭oss鐢ㄦ潵澶勭悊鏂拌繛鎺ュ姞鍏ョ殑浜嬩欢锛孨ioWorker鐢ㄦ潵澶勭悊鍚勪釜杩炴帴瀵筍ocket鐨勮鍐欎簨浠讹紝鍏朵腑Boss閫氳繃NioWorkerPool鑾峰彇NioWorker瀹炰緥锛孨etty3妯″紡浣跨敤RoundRobin鏂瑰紡鏀惧洖NioWorker瀹炰緥銆傛洿褰㈣薄涓鐐圭殑锛屽彲浠ラ氳繃Scalable IO In Java鐨勮繖寮犲浘琛ㄨ揪锛?br />
鑻ヤ笌Ractor妯″紡瀵瑰簲锛孨ioSelector涓寘鍚簡Synchronous Event Demultiplexer锛岃孋hannelPipeline涓鐞嗙潃鎵鏈塃ventHandler锛屽洜鑰孨ioSelector鍜孋hannelPipeline鍏卞悓鏋勬垚浜咺nitiation Dispatcher銆?br />5. ChannelSink锛?/strong>鍦–hannelHandler澶勭悊瀹屾垚鎵鏈夐昏緫闇瑕佸悜瀹㈡埛绔啓鍝嶅簲鏁版嵁鏃讹紝涓鑸細璋冪敤Netty Channel涓殑write鏂规硶锛岀劧鑰屽湪杩欎釜write鏂规硶瀹炵幇涓紝瀹冧笉鏄洿鎺ュ悜鍏跺唴閮ㄧ殑Socket鍐欐暟鎹紝鑰屾槸浜ょ粰Channels甯姪绫伙紝鍐呴儴鍒涘缓DownstreamMessageEvent锛屽弽鍚戜粠ChannelPipeline鐨勭閬撲腑娴佽繃鍘伙紝鐩村埌绗竴涓狢hannelHandler澶勭悊瀹屾瘯锛屾渶鍚庝氦缁機hannelSink澶勭悊锛屼互閬垮厤闃诲鍐欒屽奖鍝嶇▼搴忕殑鍚炲悙閲忋侰hannelSink灏嗚繖涓狹essageEvent鎻愪氦缁橬etty Channel涓殑writeBufferQueue锛屾渶鍚嶯ioWorker浼氱瓑鍒拌繖涓狽IO Channel宸茬粡鍙互澶勭悊鍐欎簨浠舵椂鏃犻樆濉炵殑鍚戣繖涓狽IO Channel鍐欐暟鎹傝繖灏辨槸涓婂浘鐨剆end鏄粠SubReactor鐩存帴鍑哄彂鐨勫師鍥犮?br />6. Channel锛?/strong>Netty鏈夎嚜宸辩殑Channel鎶借薄锛屽畠鏄竴涓祫婧愮殑瀹瑰櫒锛屽寘鍚簡鎵鏈変竴涓繛鎺ユ秹鍙婂埌鐨勬墍鏈夎祫婧愮殑楗敤锛屽灏佽NIO Channel銆丆hannelPipeline銆丅oss銆丯ioWorkerPool绛夈傚彟澶栧畠杩樻彁渚涗簡鍚戝唴閮∟IO Channel鍐欏搷搴旀暟鎹殑鎺ュ彛write銆佽繛鎺?缁戝畾鍒版煇涓湴鍧鐨刢onnect/bind鎺ュ彛绛夛紝涓汉鎰熻铏界劧瀵笴hannel鏈韩鏉ヨ锛屽洜涓哄畠灏佽浜哊IO Channel锛屽洜鑰岃繖浜涙帴鍙e畾涔夊湪杩欓噷鏄悎鐞嗙殑锛屼絾鏄鏋滆冭檻鍒癗etty鐨勬灦鏋勶紝瀹冪殑Channel鍙槸涓涓祫婧愬鍣紝鏈夎繖涓狢hannel瀹炰緥灏卞彲浠ュ緱鍒板拰瀹冪浉鍏崇殑鍩烘湰鎵鏈夎祫婧愶紝鍥犺岃繖绉峸rite銆乧onnect銆乥ind鍔ㄤ綔涓嶅簲璇ュ啀鐢卞畠璐熻矗锛岃屾槸搴旇鐢卞叾浠栫被鏉ヨ礋璐o紝姣斿鍦∟etty4涓氨鍦–hannelHandlerContext娣诲姞浜唚rite鏂规硶锛岃櫧鐒秐etty4骞舵病鏈夊垹闄hannel涓殑write鎺ュ彛銆?br />

Netty3涓殑Intercepting Filter妯″紡

濡傛灉璇碦eactor妯″紡鏄疦etty3鐨勯鏋讹紝閭d箞Intercepting Filter妯″紡鍒欐槸Netty鐨勪腑鏋€俁eactor妯″紡涓昏搴旂敤鍦∟etty3鐨勫唴閮ㄥ疄鐜帮紝瀹冩槸Netty3鍏锋湁鑹ソ鎬ц兘鐨勫熀纭锛岃孖ntercepting Filter妯″紡鍒欐槸ChannelHandler缁勫悎瀹炵幇涓涓簲鐢ㄧ▼搴忛昏緫鐨勫熀纭锛屽彧鏈夊緢濂界殑鐞嗚В浜嗚繖涓ā寮忔墠鑳戒娇鐢ㄥソNetty锛岀敋鑷宠兘寰楀績搴旀墜銆?br />
鍏充簬Intercepting Filter妯″紡鐨勮缁嗕粙缁嶅彲浠ュ弬鑰?a href="//www.cqk4s.cn/DLevin/archive/2015/09/03/427086.html">杩欓噷锛屾湰鑺備富瑕佷粙缁峃etty3涓Intercepting Filter妯″紡鐨勫疄鐜帮紝鍏跺疄灏辨槸DefaultChannelPipeline瀵笽ntercepting Filter妯″紡鐨勫疄鐜般傚湪涓婃枃鏈夋彁鍒癗etty3鐨凜hannelPipeline鏄疌hannelHandler鐨勫鍣紝鐢ㄤ簬瀛樺偍涓庣鐞咰hannelHandler锛屽悓鏃跺畠鍦∟etty3涓篃璧峰埌妗ユ鐨勪綔鐢紝鍗冲畠鏄繛鎺etty3鍐呴儴鍒版墍鏈塁hannelHandler鐨勬ˉ姊併備綔涓篊hannelPipeline鐨勫疄鐜拌匘efaultChannelPipeline锛屽畠浣跨敤涓涓狢hannelHandler鐨勫弻鍚戦摼琛ㄦ潵瀛樺偍锛屼互DefaultChannelPipelineContext浣滀负鑺傜偣锛?br />
public interface ChannelHandlerContext {
    Channel getChannel();

    ChannelPipeline getPipeline();

    String getName();

    ChannelHandler getHandler();

    
boolean canHandleUpstream();
    
boolean canHandleDownstream();
    
void sendUpstream(ChannelEvent e);
    
void sendDownstream(ChannelEvent e);
    Object getAttachment();

    
void setAttachment(Object attachment);
}

private final class DefaultChannelHandlerContext implements ChannelHandlerContext {
   
volatile DefaultChannelHandlerContext next;
   
volatile DefaultChannelHandlerContext prev;
   
private final String name;
   
private final ChannelHandler handler;
   
private final boolean canHandleUpstream;
   
private final boolean canHandleDownstream;
   
private volatile Object attachment;
.....
}
鍦―efaultChannelPipeline涓紝瀹冨瓨鍌ㄤ簡鍜屽綋鍓岰hannelPipeline鐩稿叧鑱旂殑Channel銆丆hannelSink浠ュ強ChannelHandler閾捐〃鐨刪ead銆乼ail锛屾墍鏈塁hannelEvent閫氳繃sendUpstream銆乻endDownstream涓哄叆鍙f祦缁忔暣涓摼琛細
public class DefaultChannelPipeline implements ChannelPipeline {
    
private volatile Channel channel;
    
private volatile ChannelSink sink;
    
private volatile DefaultChannelHandlerContext head;
    
private volatile DefaultChannelHandlerContext tail;
......
    
public void sendUpstream(ChannelEvent e) {
        DefaultChannelHandlerContext head 
= getActualUpstreamContext(this.head);
        
if (head == null) {
            
return;
        }
        sendUpstream(head, e);
    }

    
void sendUpstream(DefaultChannelHandlerContext ctx, ChannelEvent e) {
        
try {
            ((ChannelUpstreamHandler) ctx.getHandler()).handleUpstream(ctx, e);
        } 
catch (Throwable t) {
            notifyHandlerException(e, t);
        }
    }

    
public void sendDownstream(ChannelEvent e) {
        DefaultChannelHandlerContext tail 
= getActualDownstreamContext(this.tail);
        
if (tail == null) {
            
try {
                getSink().eventSunk(
this, e);
                
return;
            } 
catch (Throwable t) {
                notifyHandlerException(e, t);
                
return;
            }
        }
        sendDownstream(tail, e);
    }

    
void sendDownstream(DefaultChannelHandlerContext ctx, ChannelEvent e) {
        
if (e instanceof UpstreamMessageEvent) {
            
throw new IllegalArgumentException("cannot send an upstream event to downstream");
        }
        
try {
            ((ChannelDownstreamHandler) ctx.getHandler()).handleDownstream(ctx, e);
        } 
catch (Throwable t) {
            e.getFuture().setFailure(t);
            notifyHandlerException(e, t);
        }
    }
瀵筓pstream浜嬩欢锛屽悜鍚庢壘鍒版墍鏈夊疄鐜颁簡ChannelUpstreamHandler鎺ュ彛鐨凜hannelHandler缁勬垚閾撅紙getActualUpstreamContext()锛?/span>锛岃屽Downstream浜嬩欢锛屽悜鍓嶆壘鍒版墍鏈夊疄鐜颁簡ChannelDownstreamHandler鎺ュ彛鐨凜hannelHandler缁勬垚閾撅紙getActualDownstreamContext()锛夛細
    private DefaultChannelHandlerContext getActualUpstreamContext(DefaultChannelHandlerContext ctx) {
        
if (ctx == null) {
            
return null;
        }
        DefaultChannelHandlerContext realCtx 
= ctx;
        
while (!realCtx.canHandleUpstream()) {
            realCtx 
= realCtx.next;
            
if (realCtx == null) {
                
return null;
            }
        }
        
return realCtx;
    }
    
private DefaultChannelHandlerContext getActualDownstreamContext(DefaultChannelHandlerContext ctx) {
        
if (ctx == null) {
            
return null;
        }
        DefaultChannelHandlerContext realCtx 
= ctx;
        
while (!realCtx.canHandleDownstream()) {
            realCtx 
= realCtx.prev;
            
if (realCtx == null) {
                
return null;
            }
        }
        
return realCtx;
    }
鍦ㄥ疄闄呭疄鐜癈hannelUpstreamHandler鎴朇hannelDownstreamHandler鏃讹紝璋冪敤 ChannelHandlerContext涓殑sendUpstream鎴杝endDownstream鏂规硶灏嗘帶鍒舵祦绋嬩氦缁欎笅涓涓? ChannelUpstreamHandler鎴栦笅涓涓狢hannelDownstreamHandler锛屾垨璋冪敤Channel涓殑write鏂规硶鍙戦? 鍝嶅簲娑堟伅銆?br />
public class MyChannelUpstreamHandler implements ChannelUpstreamHandler {
    
public void handleUpstream(ChannelHandlerContext ctx, ChannelEvent e) throws Exception {
        
// handle current logic, use Channel to write response if needed.
        
// ctx.getChannel().write(message);
        ctx.sendUpstream(e);
    }
}

public class MyChannelDownstreamHandler implements ChannelDownstreamHandler {
    
public void handleDownstream(
            ChannelHandlerContext ctx, ChannelEvent e) 
throws Exception {
        
// handle current logic
        ctx.sendDownstream(e);
    }
}
褰揅hannelHandler鍚慍hannelPipelineContext鍙戦佷簨浠舵椂锛屽叾鍐呴儴浠庡綋鍓岰hannelPipelineContext鑺傜偣鍑哄彂鎵惧埌涓嬩竴涓狢hannelUpstreamHandler鎴朇hannelDownstreamHandler瀹炰緥锛屽苟鍚戝叾鍙戦丆hannelEvent锛屽浜嶥ownstream閾撅紝濡傛灉鍒拌揪閾惧熬锛屽垯灏咰hannelEvent鍙戦佺粰ChannelSink锛?br />
public void sendDownstream(ChannelEvent e) {
    DefaultChannelHandlerContext prev 
= getActualDownstreamContext(this.prev);
   
if (prev == null) {
       
try {
            getSink().eventSunk(DefaultChannelPipeline.
this, e);
        } 
catch (Throwable t) {
            notifyHandlerException(e, t);
        }
    } 
else {
        DefaultChannelPipeline.
this.sendDownstream(prev, e);
    }
}

public void sendUpstream(ChannelEvent e) {
    DefaultChannelHandlerContext next 
= getActualUpstreamContext(this.next);
   
if (next != null) {
        DefaultChannelPipeline.
this.sendUpstream(next, e);
    }
}
姝f槸鍥犱负杩欎釜瀹炵幇锛屽鏋滃湪涓涓湯灏剧殑ChannelUpstreamHandler涓厛绉婚櫎鑷繁锛屽湪鍚戞湯灏炬坊鍔犱竴涓柊鐨凜hannelUpstreamHandler锛屽畠鏄棤鏁堢殑锛屽洜涓哄畠鐨刵ext宸茬粡鍦ㄨ皟鐢ㄥ墠灏卞浐瀹氳缃负null浜嗐?br />
ChannelPipeline浣滀负ChannelHandler鐨勫鍣紝瀹冭繕鎻愪緵浜嗗悇绉嶅銆佸垹銆佹敼ChannelHandler閾捐〃涓殑鏂规硶锛岃屼笖濡傛灉鏌愪釜ChannelHandler杩樺疄鐜颁簡LifeCycleAwareChannelHandler锛屽垯璇hannelHandler鍦ㄨ娣诲姞杩汣hannelPipeline鎴栦粠涓垹闄ゆ椂閮戒細寰楀埌鍚屽織锛?br />
public interface LifeCycleAwareChannelHandler extends ChannelHandler {
    
void beforeAdd(ChannelHandlerContext ctx) throws Exception;
    
void afterAdd(ChannelHandlerContext ctx) throws Exception;
    
void beforeRemove(ChannelHandlerContext ctx) throws Exception;
    
void afterRemove(ChannelHandlerContext ctx) throws Exception;
}

public interface ChannelPipeline {
    
void addFirst(String name, ChannelHandler handler);
    
void addLast(String name, ChannelHandler handler);
    
void addBefore(String baseName, String name, ChannelHandler handler);
    
void addAfter(String baseName, String name, ChannelHandler handler);
    
void remove(ChannelHandler handler);
    ChannelHandler remove(String name);

    
<extends ChannelHandler> T remove(Class<T> handlerType);
    ChannelHandler removeFirst();

    ChannelHandler removeLast();

    
void replace(ChannelHandler oldHandler, String newName, ChannelHandler newHandler);
    ChannelHandler replace(String oldName, String newName, ChannelHandler newHandler);

    
<extends ChannelHandler> T replace(Class<T> oldHandlerType, String newName, ChannelHandler newHandler);
    ChannelHandler getFirst();

    ChannelHandler getLast();

    ChannelHandler get(String name);

    
<extends ChannelHandler> T get(Class<T> handlerType);
    ChannelHandlerContext getContext(ChannelHandler handler);

    ChannelHandlerContext getContext(String name);

    ChannelHandlerContext getContext(Class
<? extends ChannelHandler> handlerType);
    
void sendUpstream(ChannelEvent e);
    
void sendDownstream(ChannelEvent e);
    ChannelFuture execute(Runnable task);

    Channel getChannel();

    ChannelSink getSink();

    
void attach(Channel channel, ChannelSink sink);
    
boolean isAttached();
    List
<String> getNames();
    Map
<String, ChannelHandler> toMap();
}

鍦―efaultChannelPipeline鐨凜hannelHandler閾炬潯鐨勫鐞嗘祦绋嬩负锛?br />

鍙傝冿細

銆奛etty涓婚〉銆?/a>
銆奛etty婧愮爜瑙h锛堝洓锛塏etty涓嶳eactor妯″紡銆?/a>
銆奛etty浠g爜鍒嗘瀽銆?/a>
Scalable IO In Java
Intercepting Filter Pattern


DLevin 2015-09-04 09:40 鍙戣〃璇勮
]]>
Intercepting Filter妯″紡璇﹁В - ▼幸运赛车规则▼幸运赛车投注▼幸运赛车开奖结果查询▼湖南幸运赛车开奖奖金▼幸运赛车走势▼//www.cqk4s.cn/DLevin/archive/2015/09/03/427086.htmlDLevinDLevinThu, 03 Sep 2015 14:14:00 GMT//www.cqk4s.cn/DLevin/archive/2015/09/03/427086.html//www.cqk4s.cn/DLevin/comments/427086.html//www.cqk4s.cn/DLevin/archive/2015/09/03/427086.html#Feedback0//www.cqk4s.cn/DLevin/comments/commentRss/427086.html//www.cqk4s.cn/DLevin/services/trackbacks/427086.html闂鎻忚堪鍦ㄦ湇鍔″櫒缂栫▼涓紝閫氬父闇瑕佸鐞嗗绉嶄笉鍚岀殑璇锋眰锛屽湪姝e紡澶勭悊璇锋眰涔嬪墠锛岄渶瑕佸璇锋眰鍋氫竴浜涢澶勭悊锛屽锛?br />
  1. 绾綍姣忎釜Client鐨勬瘡娆¤闂俊鎭?/li>
  2. 瀵笴lient杩涜璁よ瘉鍜屾巿鏉冩鏌ワ紙Authentication and Authorization锛夈?/li>
  3. 妫鏌ュ綋鍓峉ession鏄惁鍚堟硶銆?/li>
  4. 妫鏌lient鐨処P鍦板潃鏄惁鍙俊璧栨垨涓嶅彲淇¤禆锛圛P鍦板潃鐧藉悕鍗曘侀粦鍚嶅崟锛夈?/li>
  5. 璇锋眰鏁版嵁鏄惁鍏堣瑙e帇鎴栬В鐮併?/li>
  6. 鏄惁鏀寔Client璇锋眰鐨勭被鍨嬨丅rowser鐗堟湰绛夈?/li>
  7. 娣诲姞鎬ц兘鐩戞帶淇℃伅銆?/li>
  8. 娣诲姞璋冭瘯淇℃伅銆?/li>
  9. 淇濊瘉鎵鏈夊紓甯搁兘琚纭崟鑾峰埌锛屽鏈鏂欏埌鐨勫紓甯稿仛閫氱敤澶勭悊锛岄槻姝㈢粰Client鐪嬪埌鍐呴儴鍫嗘爤淇℃伅銆?br />

鍦ㄥ搷搴旇繑鍥炵粰瀹㈡埛绔箣鍓嶏紝鏈夋椂鍊欎篃闇瑕佸仛涓浜涢澶勭悊鍐嶈繑鍥烇細

  1. 瀵瑰搷搴旀秷鎭紪鐮佹垨鍘嬬缉銆?/li>
  2. 涓烘墍鏈夊搷搴旀坊鍔犲叕鍏卞ご銆佸熬绛夋秷鎭?/li>
  3. 杩涗竴姝nrich鍝嶅簲娑堟伅锛屽娣诲姞鍏叡瀛楁銆丼ession淇℃伅銆丆ookie淇℃伅锛岀敋鑷冲畬鍏ㄦ敼鍙樺搷搴旀秷鎭瓑銆?/li>
濡備綍瀹炵幇杩欐牱鐨勯渶姹傦紝鍚屾椂淇濇寔鍙墿灞曟с佸彲閲嶇敤鎬с佸彲閰嶇疆銆佺Щ妞嶆э紵

闂瑙e喅

瑕佸疄鐜拌繖绉嶉渶姹傦紝鏈鐩磋鐨勬柟娉曞氨鏄湪姣忎釜璇锋眰澶勭悊杩囩▼涓坊鍔犳墍鏈夎繖浜涢昏緫锛屼负浜嗗噺灏戜唬鐮侀噸澶嶏紝鍙互灏嗘墍鏈夎繖浜涙鏌ユ彁鍙栨垚鏂规硶锛岃繖鏍峰湪姣忎釜澶勭悊鏂规硶涓皟鐢ㄥ嵆鍙細
public Response service1(Request request) {
    validate(request);
    request 
= transform(request);
    Response response 
= process1(request);
    
return transform(response);
}
姝ゆ椂锛屽鏋滃嚭鐜皊ervice2鏂规硶锛屼緷鐒堕渶瑕佹嫹璐漵ervice1涓殑瀹炵幇锛岀劧鍚庡皢process1鎹㈡垚process2鍗冲彲銆傝繖涓椂鍊欐垜浠彂鐜板緢澶氶噸澶嶄唬鐮侊紝缁х画瀵瑰畠閲嶆瀯锛屾瘮濡傛彁鍙栧叕鍏遍昏緫鍒板熀绫绘垚妯$増鏂规硶锛岃繖绉嶄娇鐢ㄧ户鎵跨殑鏂瑰紡浼氬紩璧峰瓙绫诲鐖剁被鐨勮﹀悎锛屽鏋滆璁╂煇浜涙ā鍧楀彉鐨勫彲閰嶇疆闇瑕佹湁澶鐨勫垽鏂昏緫锛屼唬鐮佸彉鐨勮噧鑲匡紱鍥犺屽彲浠ユ洿杩涗竴姝ワ紝灏嗘墍鏈夊鐞嗛昏緫鎶借薄鍑轰竴涓狿rocessor鎺ュ彛锛岀劧鍚庝娇鐢―ecorate妯″紡锛堝嵆寮曠敤浼樹簬缁ф壙锛夛細
public interface Processor {
    Response process(Request request);
}
public class CoreProcessor implements Processor {
    
public Response process(Request request) {
        
// do process/calculation
    }
}
public class DecoratedProcessor implements Processor {
    
private final Processor innerProcessor;
    
public DecoratedProcessor(Processor processor) {
        
this.innerProcessor = processor;
    }

    
public Response process(Request request) {
        request 
= preProcess(request);
        Response response 
= innerProcessor.process(request);
        response 
= postProcess(response);
        
return response;
    }

    
protected Request preProcess(Request request) {
        
return request;
    }
    
protected Response postProcess(Response response) {
        
return response;
    }
}

public void Transformer extends DecoratedProcessor {
    
public Transformer(Processor processor) {
        
super(processor);
    }

    
protected Request preProcess(Request request) {
        
return transformRequest(request);
    }
    
protected Response postProcess(Response response) {
        
return transformResponse(response);
    }
}
姝ゆ椂锛屽鏋滈渶瑕佸湪鐪熸鐨勫鐞嗛昏緫涔嬪墠鍔犲叆鍏朵粬鐨勯澶勭悊閫昏緫锛屽彧闇瑕佺户鎵緿ecoratedProcessor锛屽疄鐜皃reProcess鎴杙ostProcess鏂规硶锛屽垎鍒湪璇锋眰澶勭悊涔嬪墠鍜岃姹傚鐞嗕箣鍚庢í鍚戝垏鍏ヤ竴浜涢昏緫锛屼篃灏辨槸鎵璋撶殑AOP缂栫▼锛氶潰鍚戝垏闈㈢殑缂栫▼锛岀劧鍚庡彧闇瑕佹牴鎹渶姹傛瀯寤鸿繖涓摼鏉★細
Processor processor = new MissingExceptionCatcher(new Debugger(new Transformer(new CoreProcessor());
Response response 
= processor.process(request);
......
杩欏凡缁忔槸鐩稿姣旇緝濂界殑璁捐浜嗭紝姣忎釜Processor鍙渶瑕佸叧娉ㄨ嚜宸辩殑瀹炵幇閫昏緫鍗冲彲锛屼唬鐮佸彉鐨勭畝娲侊紱骞朵笖姣忎釜Processor鍚勮嚜鐙珛锛屽彲閲嶇敤鎬уソ锛屾祴璇曟柟渚匡紱鏁存潯閾句笂鑳藉疄鐜扮殑鍔熻兘鍙槸鍙栧喅浜庨摼鐨勬瀯閫狅紝鍥犺屽彧闇瑕佹湁涓绉嶆柟娉曢厤缃摼鐨勬瀯閫犲嵆鍙紝鍙厤缃т篃鍙樺緱鐏垫椿锛涚劧鑰屽緢澶氭椂鍊欏紩鐢ㄦ槸涓绉嶉潤鎬佺殑渚濊禆锛岃屾棤娉曟弧瓒冲姩鎬佺殑闇姹傘傝鏋勯犺繖鏉¢摼锛屾瘡涓墠缃甈rocessor闇瑕佺煡閬撳叾鍚庣殑Processor锛岃繖鍦ㄦ煇浜涙儏鍐典笅骞朵笉鏄湪璧峰垵灏辩煡閬撶殑銆傛鏃讹紝鎴戜滑闇瑕佸紩鍏ntercepting Filter妯″紡鏉ュ疄鐜板姩鎬佺殑鏀瑰彉鏉¢摼銆?br />

Intercepting Filter妯″紡

鍦ㄥ墠鏂囧凡缁忔瀯寤轰簡涓鏉$敱寮曠敤鑰屾垚鐨凱rocessor閾撅紝鐒惰岃繖鏄竴鏉¢潤鎬侀摼锛屽苟涓旈渶瑕佷竴寮濮嬪氨鑳芥瀯閫犲嚭杩欐潯閾撅紝涓轰簡瑙e喅杩欎釜闄愬埗锛屾垜浠彲浠ュ紩鍏ヤ竴涓狿rocessorChain鏉ョ淮鎶よ繖鏉¢摼锛屽苟涓旇繖鏉¢摼鍙互鍔ㄦ佺殑鏋勫缓銆?br />
鏈夊绉嶆柟寮忓彲浠ュ疄鐜板苟鎺у埗杩欎釜閾撅細
  1. 鍦ㄥ瓨鍌ㄤ笂锛屽彲浠ヤ娇鐢ㄦ暟缁勬潵瀛樺偍鎵鏈夌殑Processor锛孭rocessor鍦ㄦ暟缁勪腑鐨勪綅缃〃绀鸿繖涓狿rocessor鍦ㄩ摼鏉′腑鐨勪綅缃紱涔熷彲浠ョ敤閾捐〃鏉ュ瓨鍌ㄦ墍鏈夌殑Processor锛屾鏃禤rocessor鍦ㄨ繖涓摼琛ㄤ腑鐨勪綅缃嵆鏄湪閾句腑鐨勪綅缃?/li>
  2. 鍦ㄦ娊璞′笂锛屽彲浠ユ墍鏈夌殑閫昏緫閮藉皝瑁呭湪Processor涓紝涔熷彲浠ュ皢鏍稿績閫昏緫浣跨敤Processor鎶借薄锛岃屽鍥撮昏緫浣跨敤Filter鎶借薄銆?/li>
  3. 鍦ㄦ祦绋嬫帶鍒朵笂锛屼竴鑸氳繃鍦≒rocessor瀹炵幇鏂规硶涓洿鎺ヤ娇鐢≒rocessorChain瀹炰緥(閫氳繃鍙傛暟鎺哄叆)鏉ユ帶鍒舵祦绋嬶紝鍒╃敤鏂规硶璋冪敤鐨勮繘鏍堝嚭鏍堢殑鐗规у疄鐜皃reProcess()鍜宲ostProcess()澶勭悊銆?/li>
鍦ㄥ疄闄呬腑浣跨敤杩欎釜妯″紡鐨勬湁锛歋ervlet鐨凢ilter鏈哄埗銆丯etty鐨凜hannelPipeline涓丼tructs2涓殑Interceptor涓兘瀹炵幇浜嗚繖涓ā寮忋?br />

Intercepting Filter妯″紡鍦⊿ervlet鐨凢ilter涓殑瀹炵幇锛圝etty鐗堟湰锛?/h2>鍏朵腑Servlet鐨凢ilter鍦↗etty鐨勫疄鐜颁腑浣跨敤鏁扮粍瀛樺偍Filter锛孎ilter鏈熬鍙互浣跨敤Servlet瀹炰緥澶勭悊鐪熸鐨勪笟鍔¢昏緫锛屽湪娴佺▼鎺у埗涓婏紝浣跨敤FilterChain鐨刣oFilter鏂规硶鏉ュ疄鐜般傚FilterChain鍦↗etty涓殑瀹炵幇锛?br />
public void doFilter(ServletRequest request, ServletResponse response) throws IOException, ServletException
   
// pass to next filter
    if (_filter < LazyList.size(_chain)) {
        FilterHolder holder
= (FilterHolder)LazyList.get(_chain, _filter++);
        Filter filter= holder.getFilter();
        filter.doFilter(request, response, this);                   
       
return;
    }

   
// Call servlet
    HttpServletRequest srequest = (HttpServletRequest)request;
   
if (_servletHolder != null) {
        _servletHolder.handle(_baseRequest,request, response);

    }
}
杩欓噷锛宊chain瀹為檯涓婃槸涓涓狥ilter鐨凙rrayList锛岀敱FilterChain璋冪敤doFilter()鍚姩璋冪敤绗竴涓狥ilter鐨刣oFilter()鏂规硶锛屽湪瀹為檯鐨凢ilter瀹炵幇涓紝闇瑕佹墜鍔ㄧ殑璋冪敤FilterChain.doFilter()鏂规硶鏉ュ惎鍔ㄤ笅涓涓狥ilter鐨勮皟鐢紝鍒╃敤鏂规硶璋冪敤鐨勮繘鏍堝嚭鏍堢殑鐗规у疄鐜癛equest鐨刾re-process鍜孯esponse鐨刾ost-process澶勭悊銆傚鏋滀笉璋冪敤FilterChain.doFilter()鏂规硶锛屽垯琛ㄧず涓嶉渶瑕佽皟鐢ㄤ箣鍚庣殑Filter锛屾祦绋嬩粠褰撳墠Filter杩斿洖锛屽湪瀹冧箣鍓嶇殑Filter鐨凢ilterChain.doFilter()璋冪敤涔嬪悗鐨勯昏緫鍙嶅悜澶勭悊鐩村埌绗竴涓狥ilter澶勭悊瀹屾垚鑰岃繑鍥炪?br />
public class MyFilter implements Filter {
    
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
        
// pre-process ServletRequest
        chain.doFilter(request, response);
        
// post-process Servlet Response
    }
}
鏁翠釜Filter閾剧殑澶勭悊娴佺▼濡備笅锛?br />

Intercepting Filter妯″紡鍦∟etty3涓殑瀹炵幇

Netty3鍦―efaultChannelPipeline涓疄鐜颁簡Intercepting Filter妯″紡锛屽叾涓瑿hannelHandler鏄畠鐨凢ilter銆傚湪Netty3鐨凞efaultChannelPipeline涓紝浣跨敤涓涓互ChannelHandlerContext涓鸿妭鐐圭殑鍙屽悜閾捐〃鏉ュ瓨鍌–hannelHandler锛屾墍鏈夌殑妯垏闈㈤昏緫鍜屽疄闄呬笟鍔¢昏緫閮界敤ChannelHandler琛ㄨ揪锛屽湪鎺у埗娴佺▼涓婁娇鐢–hannelHandlerContext鐨剆endDownstream()鍜宻endUpstream()鏂规硶鏉ユ帶鍒舵祦绋嬨備笉鍚屼簬Servlet鐨凢ilter锛孋hannelHandler鏈変袱涓瓙鎺ュ彛锛欳hannelUpstreamHandler鍜孋hannelDownstreamHandler鍒嗗埆鐢ㄦ潵璇锋眰杩涘叆鏃剁殑澶勭悊娴佺▼鍜屽搷搴斿嚭鍘绘椂鐨勫鐞嗘祦绋嬨傚浜嶤lient鐨勮姹傦紝浠嶥efaultChannelPipeline鐨剆endUpstream()鏂规硶鍏ュ彛锛?br />
public void sendDownstream(ChannelEvent e) {
    DefaultChannelHandlerContext tail 
= getActualDownstreamContext(this.tail);
   
if (tail == null) {
       
try {
            getSink().eventSunk(
this, e);
           
return;
        } 
catch (Throwable t) {
            notifyHandlerException(e, t);
           
return;
        }
    }
    sendDownstream(tail, e);
}
void sendDownstream(DefaultChannelHandlerContext ctx, ChannelEvent e) {
   
if (e instanceof UpstreamMessageEvent) {
       
throw new IllegalArgumentException("cannot send an upstream event to downstream");
    }
   
try {
        ((ChannelDownstreamHandler) ctx.getHandler()).handleDownstream(ctx, e)
     } 
catch (Throwable t) {
        e.getFuture().setFailure(t);
        notifyHandlerException(e, t);
    }
}
濡傛灉鏈夊搷搴旀秷鎭紝璇ユ秷鎭粠DefaultChannelPipeline鐨剆endDownstream()鏂规硶涓哄叆鍙o細
public void sendUpstream(ChannelEvent e) {
    DefaultChannelHandlerContext head 
= getActualUpstreamContext(this.head);
   
if (head == null) {
        return;
    }
    sendUpstream(head, e);
}
void sendUpstream(DefaultChannelHandlerContext ctx, ChannelEvent e) {
   
try {
        ((ChannelUpstreamHandler) ctx.getHandler()).handleUpstream(ctx, e);
    } 
catch (Throwable t) {
        notifyHandlerException(e, t);
    }
}
鍦ㄥ疄闄呭疄鐜癈hannelUpstreamHandler鎴朇hannelDownstreamHandler鏃讹紝璋冪敤ChannelHandlerContext涓殑sendUpstream鎴杝endDownstream鏂规硶灏嗘帶鍒舵祦绋嬩氦缁欎笅涓涓狢hannelUpstreamHandler鎴栦笅涓涓狢hannelDownstreamHandler锛屾垨璋冪敤Channel涓殑write鏂规硶鍙戦佸搷搴旀秷鎭?br />
public class MyChannelUpstreamHandler implements ChannelUpstreamHandler {
    
public void handleUpstream(ChannelHandlerContext ctx, ChannelEvent e) throws Exception {
        
// handle current logic, use Channel to write response if needed.
        
// ctx.getChannel().write(message);
        ctx.sendUpstream(e);
    }
}

public class MyChannelDownstreamHandler implements ChannelDownstreamHandler {
    
public void handleDownstream(
            ChannelHandlerContext ctx, ChannelEvent e) 
throws Exception {
        
// handle current logic
        ctx.sendDownstream(e);
    }
}
褰揅hannelHandler鍚慍hannelPipelineContext鍙戦佷簨浠舵椂锛屽叾鍐呴儴浠庡綋鍓岰hannelPipelineContext 鑺傜偣鍑哄彂鎵惧埌涓嬩竴涓狢hannelUpstreamHandler鎴朇hannelDownstreamHandler瀹炰緥锛屽苟鍚戝叾鍙戦? ChannelEvent锛屽浜嶥ownstream閾撅紝濡傛灉鍒拌揪閾惧熬锛屽垯灏咰hannelEvent鍙戦佺粰ChannelSink锛?br />
public void sendDownstream(ChannelEvent e) {
    DefaultChannelHandlerContext prev 
= getActualDownstreamContext(this.prev);
   
if (prev == null) {
       
try {
            getSink().eventSunk(DefaultChannelPipeline.
this, e);
        } 
catch (Throwable t) {
            notifyHandlerException(e, t);
        }
    } 
else {
        DefaultChannelPipeline.
this.sendDownstream(prev, e);
    }
}

public void sendUpstream(ChannelEvent e) {
    DefaultChannelHandlerContext next 
= getActualUpstreamContext(this.next);
   
if (next != null) {
        DefaultChannelPipeline.
this.sendUpstream(next, e);
    }
}
姝f槸鍥犱负杩欎釜瀹炵幇锛屽鏋滃湪涓涓湯灏剧殑ChannelUpstreamHandler涓厛绉婚櫎鑷繁锛屽湪鍚戞湯灏炬坊鍔犱竴涓柊鐨凜hannelUpstreamHandler锛屽畠鏄棤鏁堢殑锛屽洜涓哄畠鐨刵ext宸茬粡鍦ㄨ皟鐢ㄥ墠灏卞浐瀹氳缃负null浜嗐?br />
鍦―efaultChannelPipeline鐨凜hannelHandler閾炬潯鐨勫鐞嗘祦绋嬩负锛?br />
鍦ㄨ繖涓疄鐜颁腑锛屼笉鍍廠ervlet鐨凢ilter瀹炵幇鍒╃敤鏂规硶璋冪敤鏍堢殑杩涘嚭鏍堟潵瀹屾垚pre-process鍜宲ost-process锛岃屾槸鍦ㄨ繘鍘荤殑閾惧拰鍑烘潵鐨勯摼鍚勮嚜璋冪敤handleUpstream()鍜宧andleDownstream()鏂规硶锛岃繖鏍蜂細寮曡捣璋冪敤鏍堝叾瀹炴槸涓ゆ潯閾剧殑鎬诲拰锛屽洜鑰岄渶瑕佹敞鎰忚繖鏉¢摼鐨勬婚暱搴︺傝繖鏍峰仛鐨勫ソ澶勬槸杩欐潯ChannelHandler鐨勯摼涓嶄緷璧栦簬鏂规硶璋冪敤鏍堬紝鑰屾槸鍦―efaultChannelPipeline鍐呴儴鏈韩鐨勯摼锛屽洜鑰屽湪handleUpstream()鎴杊andleDownstream()鍙互闅忔椂灏嗘墽琛屾祦绋嬭浆鍙戠粰鍏朵粬绾跨▼鎴栫嚎绋嬫睜锛屽彧闇瑕佷繚鐣機hannelPipelineContext寮曠敤锛屽湪澶勭悊瀹屾垚鍚庣敤杩欎釜ChannelPipelineContext閲嶆柊鍚戣繖鏉¢摼鐨勫悗涓涓妭鐐瑰彂閫丆hannelEvent锛岀劧鑰岀敱浜嶴ervlet鐨凢ilter渚濊禆浜庢柟娉曠殑璋冪敤鏍堬紝鍥犺屾柟娉曡繑鍥炴剰鍛崇潃鎵鏈夋墽琛屽畬鎴愶紝杩欑闄愬埗鍦ㄥ紓姝ョ紪绋嬩腑浼氬紩璧烽棶棰橈紝鍥犺孲ervlet鍦?.0鍚庡紩鍏ヤ簡Async鐨勬敮鎸併?br />

Intercepting Filter妯″紡鐨勭己鐐?/h2>绠鍗曟彁涓涓嬭繖涓ā寮忕殑缂虹偣锛?br />1. 鐩稿浼犵粺鐨勭紪绋嬫ā鍨嬶紝杩欎釜妯″紡鏈変竴瀹氱殑瀛︿範鏇茬嚎锛岄渶瑕佸緢濂界殑鐞嗚В璇ユā寮忓悗鎵嶈兘鐏垫椿鐨勫簲鐢ㄥ畠鏉ョ紪绋嬨?br />2. 闇瑕佸垝鍒嗕笉鍚岀殑閫昏緫鍒颁笉鍚岀殑Filter涓紝杩欐湁浜涙椂鍊欏苟涓嶆槸閭d箞瀹规槗銆?br />3. 鍚勪釜Filter涔嬮棿鍏变韩鏁版嵁灏嗗彉寰楀洶闅俱傚湪Netty3涓彲浠ヨ嚜瀹氫箟鑷繁鐨凜hannelEvent鏉ュ疄鐜拌嚜瀹氫箟娑堟伅鐨勪紶杈擄紝鎴栬呬娇鐢–hannelPipelineContext鐨凙ttachment瀛楁鏉ュ疄鐜版秷鎭紶杈擄紝鑰孲ervlet涓殑Filter鍒欐病鏈夋彁渚涚被浼肩殑鏈哄埗锛屽鏋滀笉鏄彲浠ラ厤缃殑鏁版嵁鍦–onfig涓紶閫掞紝鍏朵粬鏃跺欑殑鏁版嵁鍏变韩闇瑕佸叾浠栨満鍒堕厤鍚堝畬鎴愩?br />

鍙傝?/h2>Core J2EE Pattern - Intercepting Filter

DLevin 2015-09-03 22:14 鍙戣〃璇勮
]]>Reactor妯″紡璇﹁В - ▼幸运赛车规则▼幸运赛车投注▼幸运赛车开奖结果查询▼湖南幸运赛车开奖奖金▼幸运赛车走势▼//www.cqk4s.cn/DLevin/archive/2015/09/02/427045.htmlDLevinDLevinWed, 02 Sep 2015 07:14:00 GMT//www.cqk4s.cn/DLevin/archive/2015/09/02/427045.html//www.cqk4s.cn/DLevin/comments/427045.html//www.cqk4s.cn/DLevin/archive/2015/09/02/427045.html#Feedback5//www.cqk4s.cn/DLevin/comments/commentRss/427045.html//www.cqk4s.cn/DLevin/services/trackbacks/427045.html 鍓嶈

绗竴娆″惉鍒癛eactor妯″紡鏄笁骞村墠鐨勬煇涓櫄涓婏紝涓涓鍙嬬獊鐒惰窇杩囨潵闂垜浠涔堟槸Reactor妯″紡锛熸垜涓婄綉鏌ヤ簡涓涓嬶紝寰堝浜洪兘鏄粰鍑篘IO涓殑 Selector鐨勪緥瀛愶紝鑰屼笖灏辨槸NIO閲孲elector澶氳矾澶嶇敤妯″瀷锛屽彧鏄粰瀹冭捣浜嗕竴涓瘮杈僨ancy鐨勫悕瀛楄屽凡锛岃櫧鐒跺畠寮曞叆浜咵ventLoop姒? 蹇碉紝杩欏鎴戞潵璇存槸鏂扮殑姒傚康锛屼絾鏄唬鐮佸疄鐜板嵈鏄竴鏍风殑锛屽洜鑰屾垜骞舵病鏈夊緢鍦ㄦ剰杩欎釜妯″紡銆傜劧鑰屾渶杩戝紑濮嬭Netty婧愮爜锛岃孯eactor妯″紡鏄緢澶氫粙缁峃etty鐨勬枃绔犱腑琚ぇ鑲嗗浼犵殑妯″紡锛屽洜鑰屾垜鍐嶆闂嚜宸憋紝浠涔堟槸Reactor妯″紡锛熸湰鏂囧氨鏄杩欎釜闂鍏充簬鎴戠殑涓浜涚悊瑙e拰灏濊瘯鐫鏉ヨВ绛斻?br />

浠涔堟槸Reactor妯″紡

瑕佸洖绛旇繖涓棶棰橈紝棣栧厛褰撶劧鏄眰鍔〨oogle鎴朩ikipedia锛屽叾涓璚ikipedia涓婅锛?#8220;The reactor design pattern is an event handling pattern for handling service requests delivered concurrently by one or more inputs. The service handler then demultiplexes the incoming requests and dispatches them synchronously to associated request handlers.”銆備粠杩欎釜鎻忚堪涓紝鎴戜滑鐭ラ亾Reactor妯″紡棣栧厛鏄?strong>浜嬩欢椹卞姩鐨勶紝鏈変竴涓垨澶氫釜骞跺彂杈撳叆婧愶紝鏈変竴涓猄ervice Handler锛屾湁澶氫釜Request Handlers
锛涜繖涓猄ervice Handler浼氬悓姝ョ殑灏嗚緭鍏ョ殑璇锋眰锛圗vent锛夊璺鐢ㄧ殑鍒嗗彂缁欑浉搴旂殑Request Handler銆傚鏋滅敤鍥炬潵琛ㄨ揪锛?br />
浠庣粨鏋勪笂锛岃繖鏈夌偣绫讳技鐢熶骇鑰呮秷璐硅呮ā寮忥紝鍗虫湁涓涓垨澶氫釜鐢熶骇鑰呭皢浜嬩欢鏀惧叆涓涓猀ueue涓紝鑰屼竴涓垨澶氫釜娑堣垂鑰呬富鍔ㄧ殑浠庤繖涓猀ueue涓璓oll浜嬩欢鏉ュ鐞嗭紱鑰孯eactor妯″紡鍒欏苟娌℃湁Queue鏉ュ仛缂撳啿锛屾瘡褰撲竴涓狤vent杈撳叆鍒癝ervice Handler涔嬪悗锛岃Service Handler浼氫富鍔ㄧ殑鏍规嵁涓嶅悓鐨凟vent绫诲瀷灏嗗叾鍒嗗彂缁欏搴旂殑Request Handler鏉ュ鐞嗐?br />
鏇村鏈殑锛岃繖绡囨枃绔狅紙Reactor An Object Behavioral Pattern for Demultiplexing and Dispatching Handles for Synchronous Events锛変笂璇达細“The Reactor design pattern handles service requests that are delivered concurrently to an application by one or more clients. Each service in an application may consistent of several methods and is represented by a separate event handler that is responsible for dispatching service-specific requests. Dispatching of event handlers is performed by an initiation dispatcher, which manages the registered event handlers. Demultiplexing of service requests is performed by a synchronous event demultiplexer. Also known as Dispatcher, Notifier”銆傝繖娈垫弿杩板拰Wikipedia涓婄殑鎻忚堪绫讳技锛屾湁澶氫釜杈撳叆婧愶紝鏈夊涓笉鍚岀殑EventHandler锛圧equestHandler锛夋潵澶勭悊涓嶅悓鐨勮姹傦紝Initiation Dispatcher鐢ㄤ簬绠$悊EventHander锛孍ventHandler棣栧厛瑕佹敞鍐屽埌Initiation Dispatcher涓紝鐒跺悗Initiation Dispatcher鏍规嵁杈撳叆鐨凟vent鍒嗗彂缁欐敞鍐岀殑EventHandler锛涚劧鑰孖nitiation Dispatcher骞朵笉鐩戝惉Event鐨勫埌鏉ワ紝杩欎釜宸ヤ綔浜ょ粰Synchronous Event Demultiplexer鏉ュ鐞嗐?br />

Reactor妯″紡缁撴瀯

鍦ㄨВ鍐充簡浠涔堟槸Reactor妯″紡鍚庯紝鎴戜滑鏉ョ湅鐪婻eactor妯″紡鏄敱浠涔堟ā鍧楁瀯鎴愩傚浘鏄竴绉嶆瘮杈冪畝娲佸舰璞$殑琛ㄧ幇鏂瑰紡锛屽洜鑰屽厛涓婁竴寮犲浘鏉ヨ〃杈惧悇涓ā鍧楃殑鍚嶇О鍜屼粬浠箣闂寸殑鍏崇郴锛?br />
Handle锛?/strong>鍗虫搷浣滅郴缁熶腑鐨勫彞鏌勶紝鏄璧勬簮鍦ㄦ搷浣滅郴缁熷眰闈笂鐨勪竴绉嶆娊璞★紝瀹冨彲浠ユ槸鎵撳紑鐨勬枃浠躲佷竴涓繛鎺?Socket)銆乀imer绛夈傜敱浜嶳eactor妯″紡涓鑸娇鐢ㄥ湪缃戠粶缂栫▼涓紝鍥犺岃繖閲屼竴鑸寚Socket Handle锛屽嵆涓涓綉缁滆繛鎺ワ紙Connection锛屽湪Java NIO涓殑Channel锛夈傝繖涓狢hannel娉ㄥ唽鍒癝ynchronous Event Demultiplexer涓紝浠ョ洃鍚琀andle涓彂鐢熺殑浜嬩欢锛屽ServerSocketChannnel鍙互鏄疌ONNECT浜嬩欢锛屽SocketChannel鍙互鏄疪EAD銆乄RITE銆丆LOSE浜嬩欢绛夈?br />Synchronous Event Demultiplexer锛?/strong>闃诲绛夊緟涓绯诲垪鐨凥andle涓殑浜嬩欢鍒版潵锛屽鏋滈樆濉炵瓑寰呰繑鍥烇紝鍗宠〃绀哄湪杩斿洖鐨凥andle涓彲浠ヤ笉闃诲鐨勬墽琛岃繑鍥炵殑浜嬩欢绫诲瀷銆傝繖涓ā鍧椾竴鑸娇鐢ㄦ搷浣滅郴缁熺殑select鏉ュ疄鐜般傚湪Java NIO涓敤Selector鏉ュ皝瑁咃紝褰揝elector.select()杩斿洖鏃讹紝鍙互璋冪敤Selector鐨剆electedKeys()鏂规硶鑾峰彇Set<SelectionKey>锛屼竴涓猄electionKey琛ㄨ揪涓涓湁浜嬩欢鍙戠敓鐨凜hannel浠ュ強璇hannel涓婄殑浜嬩欢绫诲瀷銆備笂鍥剧殑“Synchronous Event Demultiplexer ---notifies--> Handle”鐨勬祦绋嬪鏋滄槸瀵圭殑锛岄偅鍐呴儴瀹炵幇搴旇鏄痵elect()鏂规硶鍦ㄤ簨浠跺埌鏉ュ悗浼氬厛璁剧疆Handle鐨勭姸鎬侊紝鐒跺悗杩斿洖銆備笉浜嗚В鍐呴儴瀹炵幇鏈哄埗锛屽洜鑰屼繚鐣欏師鍥俱?br />Initiation Dispatcher锛?/strong>鐢ㄤ簬绠$悊Event Handler锛屽嵆EventHandler鐨勫鍣紝鐢ㄤ互娉ㄥ唽銆佺Щ闄ventHandler绛夛紱鍙﹀锛屽畠杩樹綔涓篟eactor妯″紡鐨勫叆鍙h皟鐢⊿ynchronous Event Demultiplexer鐨剆elect鏂规硶浠ラ樆濉炵瓑寰呬簨浠惰繑鍥烇紝褰撻樆濉炵瓑寰呰繑鍥炴椂锛屾牴鎹簨浠跺彂鐢熺殑Handle灏嗗叾鍒嗗彂缁欏搴旂殑Event Handler澶勭悊锛屽嵆鍥炶皟EventHandler涓殑handle_event()鏂规硶銆?br />Event Handler锛?/strong>瀹氫箟浜嬩欢澶勭悊鏂规硶锛歨andle_event()锛屼互渚汭nitiationDispatcher鍥炶皟浣跨敤銆?br />Concrete Event Handler锛?/strong>浜嬩欢EventHandler鎺ュ彛锛屽疄鐜扮壒瀹氫簨浠跺鐞嗛昏緫銆?br />

Reactor妯″紡妯″潡涔嬮棿鐨勪氦浜?/h2> 绠鍗曟弿杩颁竴涓婻eactor鍚勪釜妯″潡涔嬮棿鐨勪氦浜掓祦绋嬶紝鍏堜粠搴忓垪鍥惧紑濮嬶細

1. 鍒濆鍖朓nitiationDispatcher锛屽苟鍒濆鍖栦竴涓狧andle鍒癊ventHandler鐨凪ap銆?br />2. 娉ㄥ唽EventHandler鍒癐nitiationDispatcher涓紝姣忎釜EventHandler鍖呭惈瀵圭浉搴擧andle鐨勫紩鐢紝浠庤屽缓绔婬andle鍒癊ventHandler鐨勬槧灏勶紙Map锛夈?br />3. 璋冪敤InitiationDispatcher鐨刪andle_events()鏂规硶浠ュ惎鍔‥vent Loop銆傚湪Event Loop涓紝璋冪敤select()鏂规硶锛圫ynchronous Event Demultiplexer锛夐樆濉炵瓑寰匛vent鍙戠敓銆?br />4. 褰撴煇涓垨鏌愪簺Handle鐨凟vent鍙戠敓鍚庯紝select()鏂规硶杩斿洖锛孖nitiationDispatcher鏍规嵁杩斿洖鐨凥andle鎵惧埌娉ㄥ唽鐨凟ventHandler锛屽苟鍥炶皟璇ventHandler鐨刪andle_events()鏂规硶銆?br />5. 鍦‥ventHandler鐨刪andle_events()鏂规硶涓繕鍙互鍚慖nitiationDispatcher涓敞鍐屾柊鐨凟venthandler锛屾瘮濡傚AcceptorEventHandler鏉ワ紝褰撴湁鏂扮殑client杩炴帴鏃讹紝瀹冧細浜х敓鏂扮殑EventHandler浠ュ鐞嗘柊鐨勮繛鎺ワ紝骞舵敞鍐屽埌InitiationDispatcher涓?br />

Reactor妯″紡瀹炵幇

鍦?a >Reactor An Object Behavioral Pattern for Demultiplexing and Dispatching Handles for Synchronous Events涓紝涓鐩翠互Logging Server鏉ュ垎鏋怰eactor妯″紡锛岃繖涓狶ogging Server鐨勫疄鐜板畬鍏ㄩ伒寰繖閲屽Reactor鎻忚堪锛屽洜鑰屾斁鍦ㄨ繖閲屼互鍋氬弬鑰冦侺ogging Server涓殑Reactor妯″紡瀹炵幇鍒嗕袱涓儴鍒嗭細Client杩炴帴鍒癓ogging Server鍜孋lient鍚慙ogging Server鍐橪og銆傚洜鑰屽瀹冪殑鎻忚堪鍒嗘垚杩欎袱涓楠ゃ?br />Client杩炴帴鍒癓ogging Server

1. Logging Server娉ㄥ唽LoggingAcceptor鍒癐nitiationDispatcher銆?br />2. Logging Server璋冪敤InitiationDispatcher鐨刪andle_events()鏂规硶鍚姩銆?br />3. InitiationDispatcher鍐呴儴璋冪敤select()鏂规硶锛圫ynchronous Event Demultiplexer锛夛紝闃诲绛夊緟Client杩炴帴銆?br />4. Client杩炴帴鍒癓ogging Server銆?br />5. InitiationDisptcher涓殑select()鏂规硶杩斿洖锛屽苟閫氱煡LoggingAcceptor鏈夋柊鐨勮繛鎺ュ埌鏉ャ?
6. LoggingAcceptor璋冪敤accept鏂规硶accept杩欎釜鏂拌繛鎺ャ?br />7. LoggingAcceptor鍒涘缓鏂扮殑LoggingHandler銆?br />8. 鏂扮殑LoggingHandler娉ㄥ唽鍒癐nitiationDispatcher涓?鍚屾椂涔熸敞鍐屽埌Synchonous Event Demultiplexer涓?锛岀瓑寰匔lient鍙戣捣鍐檒og璇锋眰銆?br />Client鍚慙ogging Server鍐橪og

1. Client鍙戦乴og鍒癓ogging server銆?br />2. InitiationDispatcher鐩戞祴鍒扮浉搴旂殑Handle涓湁浜嬩欢鍙戠敓锛岃繑鍥為樆濉炵瓑寰咃紝鏍规嵁杩斿洖鐨凥andle鎵惧埌LoggingHandler锛屽苟鍥炶皟LoggingHandler涓殑handle_event()鏂规硶銆?br />3. LoggingHandler涓殑handle_event()鏂规硶涓鍙朒andle涓殑log淇℃伅銆?br />4. 灏嗘帴鏀跺埌鐨刲og鍐欏叆鍒版棩蹇楁枃浠躲佹暟鎹簱绛夎澶囦腑銆?br />3.4姝ラ寰幆鐩村埌褰撳墠鏃ュ織澶勭悊瀹屾垚銆?br />5. 杩斿洖鍒癐nitiationDispatcher绛夊緟涓嬩竴娆℃棩蹇楀啓璇锋眰銆?br />
鍦?a >Reactor An Object Behavioral Pattern for Demultiplexing and Dispatching Handles for Synchronous Events鏈夊Reactor妯″紡鐨凜++鐨勫疄鐜扮増鏈紝澶氬勾涓嶇敤C++锛屽洜鑰岀暐杩囥?nbsp;

Java NIO瀵筊eactor鐨勫疄鐜?/h2>鍦↗ava鐨凬IO涓紝瀵筊eactor妯″紡鏈夋棤缂濈殑鏀寔锛屽嵆浣跨敤Selector绫诲皝瑁呬簡鎿嶄綔绯荤粺鎻愪緵鐨凷ynchronous Event Demultiplexer鍔熻兘銆傝繖涓狣oug Lea宸茬粡鍦?a >Scalable IO In Java涓湁闈炲父娣卞叆鐨勮В閲婁簡锛屽洜鑰屼笉鍐嶈禈杩帮紝鍙﹀杩欑瘒鏂囩珷瀵笵oug Lea鐨?a >Scalable IO In Java鏈変竴浜涚畝鍗曡В閲婏紝鑷冲皯瀹冪殑浠g爜鏍煎紡姣擠oug Lea鐨凱PT瑕佹暣娲佷竴浜涖?br />
闇瑕佹寚鍑虹殑鏄紝涓嶅悓杩欓噷浣跨敤InitiationDispatcher鏉ョ鐞咵ventHandler锛屽湪Doug Lea鐨勭増鏈腑浣跨敤SelectionKey涓殑Attachment鏉ュ瓨鍌ㄥ搴旂殑EventHandler锛屽洜鑰屼笉闇瑕佹敞鍐孍ventHandler杩欎釜姝ラ锛屾垨鑰呰缃瓵ttachment灏辨槸杩欓噷鐨勬敞鍐屻傝屼笖鍦ㄨ繖绡囨枃绔犱腑锛孌oug Lea浠庡崟绾跨▼鐨凴eactor銆丄cceptor銆丠andler瀹炵幇杩欎釜妯″紡鍑哄彂锛涙紨鍖栦负灏咹andler涓殑澶勭悊閫昏緫澶氱嚎绋嬪寲锛屽疄鐜扮被浼糚roactor妯″紡锛屾鏃舵墍鏈夌殑IO鎿嶄綔杩樻槸鍗曠嚎绋嬬殑锛屽洜鑰屽啀婕斿寲鍑轰竴涓狹ain Reactor鏉ュ鐞咰ONNECT浜嬩欢(Acceptor)锛岃屽涓猄ub Reactor鏉ュ鐞哛EAD銆乄RITE绛変簨浠?Handler)锛岃繖浜汼ub Reactor鍙互鍒嗗埆鍐嶈嚜宸辩殑绾跨▼涓墽琛岋紝浠庤孖O鎿嶄綔涔熷绾跨▼鍖栥傝繖涓渶鍚庝竴涓ā鍨嬫鏄疦etty涓娇鐢ㄧ殑妯″瀷銆傚苟涓斿湪Reactor An Object Behavioral Pattern for Demultiplexing and Dispatching Handles for Synchronous Events鐨?.5 Determine the Number of Initiation Dispatchers in an Application涓篃鏈夌浉搴旂殑鎻忚堪銆?br />

EventHandler鎺ュ彛瀹氫箟

瀵笶ventHandler鐨勫畾涔夋湁涓ょ璁捐鎬濊矾锛歴ingle-method璁捐鍜宮ulti-method璁捐锛?br />A single-method interface锛?/strong>瀹冨皢Event灏佽鎴愪竴涓狤vent Object锛孍ventHandler鍙畾涔変竴涓猦andle_event(Event event)鏂规硶銆傝繖绉嶈璁$殑濂藉鏄湁鍒╀簬鎵╁睍锛屽彲浠ュ悗鏉ユ柟渚跨殑娣诲姞鏂扮殑Event绫诲瀷锛岀劧鑰屽湪瀛愮被鐨勫疄鐜颁腑锛岄渶瑕佸垽鏂笉鍚岀殑Event绫诲瀷鑰屽啀娆℃墿灞曟垚 涓嶅悓鐨勫鐞嗘柟娉曪紝浠庤繖涓搴︿笂鏉ヨ锛屽畠鍙堜笉鍒╀簬鎵╁睍銆傚彟澶栧湪Netty3鐨勪娇鐢ㄨ繃绋嬩腑锛岀敱浜庡畠涓嶅仠鐨勫垱寤篊hannelEvent绫伙紝鍥犺屼細寮曡捣GC鐨勪笉绋冲畾銆?br />A multi-method interface锛?/strong>杩欑璁捐鏄皢涓嶅悓鐨凟vent绫诲瀷鍦? EventHandler涓畾涔夌浉搴旂殑鏂规硶銆傝繖绉嶈璁″氨鏄疦etty4涓娇鐢ㄧ殑绛栫暐锛屽叾涓竴涓洰鐨勬槸閬垮厤ChannelEvent鍒涘缓寮曡捣鐨凣C涓嶇ǔ瀹氾紝 鍙﹀涓涓ソ澶勬槸瀹冨彲浠ラ伩鍏嶅湪EventHandler瀹炵幇鏃跺垽鏂笉鍚岀殑Event绫诲瀷鑰屾湁涓嶅悓鐨勫疄鐜帮紝鐒惰岃繖绉嶈璁′細缁欐墿灞曟柊鐨凟vent绫诲瀷鏃跺甫鏉ラ潪甯? 澶х殑楹荤儲锛屽洜涓哄畠闇瑕佽鎺ュ彛銆?br />
鍏充簬Netty4瀵筃etty3鐨勬敼杩涘彲浠ュ弬鑰?a >杩欓噷锛?br />
ChannelHandler with no event objectIn 3.x, every I/O operation created a ChannelEvent object. For each read / write, it additionally created a new ChannelBuffer. It simplified the internals of Netty quite a lot because it delegates resource management and buffer pooling to the JVM. However, it often was the root cause of GC pressure and uncertainty which are sometimes observed in a Netty-based application under high load.

4.0 removes event object creation almost completely by replacing the event objects with strongly typed method invocations. 3.x had catch-all event handler methods such as handleUpstream() and handleDownstream(), but this is not the case anymore. Every event type has its own handler method now:

涓轰粈涔堜娇鐢≧eactor妯″紡

褰掑姛涓嶯etty鍜孞ava NIO瀵筊eactor鐨勫浼狅紝鏈枃鎱曞悕鑰屽涔犵殑Reactor妯″紡锛屽洜鑰屽凡缁忛粯璁eactor鍏锋湁闈炲父浼樼鐨勬ц兘锛岀劧鑰屾厱鍚嶅綊鎱曞悕锛屽埌杩欓噷锛屾垜杩樻槸瑕佷笉寰椾笉闂嚜宸盧eactor妯″紡鐨勫ソ澶勫湪鍝噷锛熷嵆涓轰粈涔堣浣跨敤杩欎釜Reactor妯″紡锛熷湪Reactor An Object Behavioral Pattern for Demultiplexing and Dispatching Handles for Synchronous Events涓槸杩欎箞璇寸殑锛?br />
Reactor Pattern浼樼偣

Separation of concerns: The Reactor pattern decouples application-independent demultiplexing and dispatching mechanisms from application-specific hook method functionality. The application-independent mechanisms become reusable components that know how to demultiplex events and dispatch the appropriate hook methods defined by Event Handlers. In contrast, the application-specific functionality in a hook method knows how to perform a particular type of service.

Improve modularity, reusability, and configurability of event-driven applications: The pattern decouples application functionality into separate classes. For instance, there are two separate classes in the logging server: one for establishing connections and another for receiving and processing logging records. This decoupling enables the reuse of the connection establishment class for different types of connection-oriented services (such as file transfer, remote login, and video-on-demand). Therefore, modifying or extending the functionality of the logging server only affects the implementation of the logging handler class.

Improves application portability: The Initiation Dispatcher’s interface can be reused independently of the OS system calls that perform event demultiplexing. These system calls detect and report the occurrence of one or more events that may occur simultaneously on multiple sources of events. Common sources of events may in- clude I/O handles, timers, and synchronization objects. On UNIX platforms, the event demultiplexing system calls are called select and poll [1]. In the Win32 API [16], the WaitForMultipleObjects system call performs event demultiplexing.

Provides coarse-grained concurrency control: The Reactor pattern serializes the invocation of event handlers at the level of event demultiplexing and dispatching within a process or thread. Serialization at the Initiation Dispatcher level often eliminates the need for more complicated synchronization or locking within an application process.

杩欎簺璨屼技鏄緢澶氭ā寮忕殑鍏辨э細瑙h︺佹彁鍗囧鐢ㄦс佹ā鍧楀寲銆佸彲绉绘鎬с佷簨浠堕┍鍔ㄣ佺粏鍔涘害鐨勫苟鍙戞帶鍒剁瓑锛屽洜鑰屽苟涓嶈兘寰堝ソ鐨勮鏄庝粈涔堬紝鐗瑰埆鏄畠榧撳惞鐨勫鎬ц兘鐨勬彁鍗囷紝杩欓噷骞舵病鏈変綋鐜板嚭鏉ャ傚綋鐒跺湪杩欑瘒鏂囩珷鐨勫紑澶存湁鎻忚堪杩囧彟涓绉嶇洿瑙傜殑瀹炵幇锛歍hread-Per-Connection锛屽嵆浼犵粺鐨勫疄鐜帮紝鎻愬埌浜嗚繖涓紶缁熷疄鐜扮殑浠ヤ笅闂锛?br />
Thread Per Connection缂虹偣

Efficiency: Threading may lead to poor performance due to context switching, synchronization, and data movement [2];

Programming simplicity: Threading may require complex concurrency control schemes;

Portability: Threading is not available on all OS platforms.
瀵逛簬鎬ц兘锛屽畠鍏跺疄灏辨槸绗竴鐐瑰叧浜嶦fficiency鐨勬弿杩帮紝鍗崇嚎绋嬬殑鍒囨崲銆佸悓姝ャ佹暟鎹殑绉诲姩浼氬紩璧锋ц兘闂銆備篃灏辨槸璇翠粠鎬ц兘鐨勮搴︿笂锛屽畠鏈澶х殑鎻愬崌灏辨槸鍑忓皯浜嗘ц兘鐨勪娇鐢紝鍗充笉闇瑕佹瘡涓狢lient瀵瑰簲涓涓嚎绋嬨傛垜鐨勭悊瑙o紝鍏朵粬涓氬姟閫昏緫澶勭悊寰堝鏃跺欎篃浼氱敤鍒扮浉鍚岀殑绾跨▼锛孖O璇诲啓鎿嶄綔鐩稿CPU鐨勬搷浣滆繕鏄鎱㈠緢澶氾紝鍗充娇Reactor鏈哄埗涓瘡娆¤鍐欏凡缁忚兘淇濊瘉闈為樆濉炶鍐欙紝杩欓噷鍙互鍑忓皯涓浜涚嚎绋嬬殑浣跨敤锛屼絾鏄繖鍑忓皯鐨勭嚎绋嬩娇鐢ㄥ鎬ц兘鏈夐偅涔堝ぇ鐨勫奖鍝嶅悧锛熺瓟妗堣矊浼兼槸鑲畾鐨勶紝杩欑瘒璁烘枃(SEDA: Staged Event-Driven Architecture - An Architecture for Well-Conditioned, Scalable Internet Service)瀵归殢鐫绾跨▼鐨勫闀垮甫鏉ユц兘闄嶄綆鍋氫簡涓涓粺璁★細

鍦ㄨ繖涓粺璁′腑锛屾瘡涓嚎绋嬩粠纾佺洏涓8KB鏁版嵁锛屾瘡涓嚎绋嬭鍚屼竴涓枃浠讹紝鍥犺屾暟鎹湰韬槸缂撳瓨鍦ㄦ搷浣滅郴缁熷唴閮ㄧ殑锛屽嵆鍑忓皯IO鐨勫奖鍝嶏紱鎵鏈夌嚎绋嬫槸浜嬪厛鍒嗛厤鐨勶紝涓嶄細鏈夌嚎绋嬪惎鍔ㄧ殑褰卞搷锛涙墍鏈変换鍔″湪娴嬭瘯鍐呴儴浜х敓锛屽洜鑰屼笉浼氭湁缃戠粶鐨勫奖鍝嶃傝缁熻鏁版嵁杩愯鐜锛歀inux 2.2.14锛?GB鍐呭瓨锛?-way 500MHz Pentium III銆備粠鍥句腑鍙互鐪嬪嚭锛岄殢鐫绾跨▼鐨勫闀匡紝鍚炲悙閲忓湪绾跨▼鏁颁负8涓乏鍙崇殑鏃跺欏紑濮嬬嚎鎬т笅闄嶏紝骞朵笖鍒?4涓互鍚庤岃繀閫熶笅闄嶏紝鍏剁浉搴斾簨浠朵篃鍦ㄧ嚎绋嬭揪鍒?56涓悗鎸囨暟涓婂崌銆傚嵆1+1<2锛屽洜涓虹嚎绋嬪垏鎹€佸悓姝ャ佹暟鎹Щ鍔ㄤ細鏈夋ц兘鎹熷け锛岀嚎绋嬫暟澧炲姞鍒颁竴瀹氭暟閲忔椂锛岃繖绉嶆ц兘褰卞搷鏁堟灉浼氭洿鍔犳槑鏄俱?br />
瀵逛簬杩欑偣锛岃繕鍙互鍙傝?a >C10K Problem锛岀敤浠ユ弿杩板悓鏃舵湁10K涓狢lient鍙戣捣杩炴帴鐨勯棶棰橈紝鍒?010骞寸殑鏃跺欏凡缁忓嚭鐜?0M Problem浜嗐?br />
褰撶劧涔熸湁浜鸿锛?a >Threads are expensive are no longer valid.鍦ㄤ笉涔呯殑灏嗘潵鍙兘鍙堜細鍙戠敓涓嶅悓鐨勫彉鍖栵紝鎴栬呰繖涓彉鍖栨鍦ㄣ佸凡缁忓彂鐢熺潃锛熸病鏈夊仛杩囨瘮杈冧粩缁嗙殑娴嬭瘯锛屽洜鑰屼笉鏁㈤殢渚挎柇瑷浠涔堬紝鐒惰屾湰浜鸿鐐癸紝鍗充娇绾跨▼鍙樼殑褰卞搷骞舵病鏈変互鍓嶉偅涔堝ぇ锛屼娇鐢≧eactor妯″紡锛岀敋鑷虫椂SEDA妯″紡鏉ュ噺灏戠嚎绋嬬殑浣跨敤锛屽啀鍔犱笂鍏朵粬瑙h︺佹ā鍧楀寲銆佹彁鍗囧鐢ㄦх瓑浼樼偣锛岃繕鏄煎緱浣跨敤鐨勩?br />

Reactor妯″紡鐨勭己鐐?/h2>Reactor妯″紡鐨勭己鐐硅矊浼间篃鏄樉鑰屾槗瑙佺殑锛?br />1. 鐩告瘮浼犵粺鐨勭畝鍗曟ā鍨嬶紝Reactor澧炲姞浜嗕竴瀹氱殑澶嶆潅鎬э紝鍥犺屾湁涓瀹氱殑闂ㄦ锛屽苟涓斾笉鏄撲簬璋冭瘯銆?br />2. Reactor妯″紡闇瑕佸簳灞傜殑Synchronous Event Demultiplexer鏀寔锛屾瘮濡侸ava涓殑Selector鏀寔锛屾搷浣滅郴缁熺殑select绯荤粺璋冪敤鏀寔锛屽鏋滆鑷繁瀹炵幇Synchronous Event Demultiplexer鍙兘涓嶄細鏈夐偅涔堥珮鏁堛?br />3. Reactor妯″紡鍦↖O璇诲啓鏁版嵁鏃惰繕鏄湪鍚屼竴涓嚎绋嬩腑瀹炵幇鐨勶紝鍗充娇浣跨敤澶氫釜Reactor鏈哄埗鐨勬儏鍐典笅锛岄偅浜涘叡浜竴涓猂eactor鐨凜hannel濡傛灉鍑虹幇涓涓暱鏃堕棿鐨勬暟鎹鍐欙紝浼氬奖鍝嶈繖涓猂eactor涓叾浠朇hannel鐨勭浉搴旀椂闂达紝姣斿鍦ㄥぇ鏂囦欢浼犺緭鏃讹紝IO鎿嶄綔灏变細褰卞搷鍏朵粬Client鐨勭浉搴旀椂闂达紝鍥犺屽杩欑鎿嶄綔锛屼娇鐢ㄤ紶缁熺殑Thread-Per-Connection鎴栬鏄竴涓洿濂界殑閫夋嫨锛屾垨鍒欐鏃朵娇鐢≒roactor妯″紡銆?br />

鍙傝?/h2> Reactor Pattern WikiPedia
Reactor An Object Behavioral Pattern for Demultiplexing and Dispatching Handles for Synchronous Events
Scalable IO In Java
C10K Problem WikiPedia


DLevin 2015-09-02 15:14 鍙戣〃璇勮
]]>娣卞叆HBase鏋舵瀯瑙f瀽锛堜簩锛?/title><link>//www.cqk4s.cn/DLevin/archive/2015/08/22/426950.html</link><dc:creator>DLevin</dc:creator><author>DLevin</author><pubDate>Sat, 22 Aug 2015 11:40:00 GMT</pubDate><guid>//www.cqk4s.cn/DLevin/archive/2015/08/22/426950.html</guid><wfw:comment>//www.cqk4s.cn/DLevin/comments/426950.html</wfw:comment><comments>//www.cqk4s.cn/DLevin/archive/2015/08/22/426950.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>//www.cqk4s.cn/DLevin/comments/commentRss/426950.html</wfw:commentRss><trackback:ping>//www.cqk4s.cn/DLevin/services/trackbacks/426950.html</trackback:ping><description><![CDATA[<h2 class="vpdh"> 鍓嶈█</h2>杩欐槸<a href="//www.cqk4s.cn/DLevin/archive/2015/08/22/426877.html">銆婃繁鍏Base鏋舵瀯瑙f瀽锛堜竴锛夈?/a>鐨勭画锛屼笉澶氬簾璇濓紝缁х画銆傘傘傘?br /><h2 class="vpdh">HBase璇荤殑瀹炵幇</h2>閫氳繃鍓嶆枃鐨勬弿杩帮紝鎴戜滑鐭ラ亾鍦℉Base鍐欐椂锛岀浉鍚孋ell(RowKey/ColumnFamily/Column鐩稿悓)骞朵笉淇濊瘉鍦ㄤ竴璧凤紝鐢氳嚦鍒犻櫎涓涓狢ell涔熷彧鏄啓鍏ヤ竴涓柊鐨凜ell锛屽畠鍚湁Delete鏍囪锛岃屼笉涓瀹氬皢涓涓狢ell鐪熸鍒犻櫎浜嗭紝鍥犺岃繖灏卞紩璧蜂簡涓涓棶棰橈紝濡備綍瀹炵幇璇荤殑闂锛熻瑙e喅杩欎釜闂锛屾垜浠厛鏉ュ垎鏋愪竴涓嬬浉鍚岀殑Cell鍙兘瀛樺湪鐨勪綅缃細棣栧厛瀵规柊鍐欏叆鐨凜ell锛屽畠浼氬瓨鍦ㄤ簬MemStore涓紱鐒跺悗瀵逛箣鍓嶅凡缁廎lush鍒癏DFS涓殑Cell锛屽畠浼氬瓨鍦ㄤ簬鏌愪釜鎴栨煇浜汼toreFile(HFile)涓紱鏈鍚庯紝瀵瑰垰璇诲彇杩囩殑Cell锛屽畠鍙兘瀛樺湪浜嶣lockCache涓傛棦鐒剁浉鍚岀殑Cell鍙兘瀛樺偍鍦ㄤ笁涓湴鏂癸紝鍦ㄨ鍙栫殑鏃跺欏彧闇瑕佹壂鐬勮繖涓変釜鍦版柟锛岀劧鍚庡皢缁撴灉鍚堝苟鍗冲彲(Merge Read)锛屽湪HBase涓壂鐬勭殑椤哄簭渚濇鏄細BlockCache銆丮emStore銆丼toreFile(HFile)銆傚叾涓璖toreFile鐨勬壂鐬勫厛浼氫娇鐢˙loom Filter杩囨护閭d簺涓嶅彲鑳界鍚堟潯浠剁殑HFile锛岀劧鍚庝娇鐢˙lock Index蹇熷畾浣岰ell锛屽苟灏嗗叾鍔犺浇鍒癇lockCache涓紝鐒跺悗浠嶣lockCache涓鍙栥傛垜浠煡閬撲竴涓狧Store鍙兘瀛樺湪澶氫釜StoreFile(HFile)锛屾鏃堕渶瑕佹壂鐬勫涓狧File锛屽鏋淗File杩囧鍙堟槸浼氬紩璧锋ц兘闂銆?br /><img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig16.png" height="278" width="769" /><br /><h2 class="vpdh">Compaction</h2>MemStore姣忔Flush浼氬垱寤烘柊鐨凥File锛岃岃繃澶氱殑HFile浼氬紩璧疯鐨勬ц兘闂锛岄偅涔堝浣曡В鍐宠繖涓棶棰樺憿锛烪Base閲囩敤Compaction鏈哄埗鏉ヨВ鍐宠繖涓棶棰橈紝鏈夌偣绫讳技Java涓殑GC鏈哄埗锛岃捣鍒滼ava涓嶅仠鐨勭敵璇峰唴瀛樿屼笉閲婃斁锛屽鍔犳ц兘锛岀劧鑰屽ぉ涓嬫病鏈夊厤璐圭殑鍗堥锛屾渶缁堟垜浠繕鏄鍦ㄦ煇涓潯浠朵笅鍘绘敹闆嗗瀮鍦撅紝寰堝鏃跺欓渶瑕丼top-The-World锛岃繖绉峉top-The-World鏈変簺鏃跺欎篃浼氬紩璧峰緢澶х殑闂锛屾瘮濡傚弬鑰冩湰浜哄啓鐨?a href="//www.cqk4s.cn/DLevin/archive/2015/08/01/426418.html">杩欑瘒鏂囩珷</a>锛屽洜鑰岃璁℃槸涓绉嶆潈琛★紝娌℃湁瀹岀編鐨勩傝繕鏄被浼糐ava涓殑GC锛屽湪HBase涓瑿ompaction鍒嗕负涓ょ锛歁inor Compaction鍜孧ajor Compaction銆?br /><ol><li class="vpdh">Minor Compaction鏄寚閫夊彇涓浜涘皬鐨勩佺浉閭荤殑StoreFile灏嗕粬浠悎骞舵垚涓涓洿澶х殑StoreFile锛屽湪杩欎釜杩囩▼涓笉浼氬鐞嗗凡缁廌eleted鎴朎xpired鐨凜ell銆備竴娆inor Compaction鐨勭粨鏋滄槸鏇村皯骞朵笖鏇村ぇ鐨凷toreFile銆傦紙杩欎釜鏄鐨勫悧锛烞igTable涓槸杩欐牱鎻忚堪Minor Compaction鐨?span style="font-size: 10.000000pt; font-family: 'Times'">锛欰s write operations execute, the size of the memtable in- creases. When the memtable size reaches a threshold, the memtable is frozen, a new memtable is created, and the frozen memtable is converted to an SSTable and written to GFS. This </span><span style="font-size: 10.000000pt; font-family: 'Times'; font-style: italic">minor compaction </span><span style="font-size: 10.000000pt; font-family: 'Times'">process has two goals: it shrinks the memory usage of the tablet server, and it reduces the amount of data that has to be read from the commit log during recovery if this server dies. Incom- ing read and write operations can continue while com- pactions occur. </span>涔熷氨鏄瀹冨皢memtable鐨勬暟鎹甪lush鐨勪竴涓狧File/SSTable绉颁负涓娆inor Compaction锛?/li><li class="vpdh">Major Compaction鏄寚灏嗘墍鏈夌殑StoreFile鍚堝苟鎴愪竴涓猄toreFile锛屽湪杩欎釜杩囩▼涓紝鏍囪涓篋eleted鐨凜ell浼氳鍒犻櫎锛岃岄偅浜涘凡缁廍xpired鐨凜ell浼氳涓㈠純锛岄偅浜涘凡缁忚秴杩囨渶澶氱増鏈暟鐨凜ell浼氳涓㈠純銆備竴娆ajor Compaction鐨勭粨鏋滄槸涓涓狧Store鍙湁涓涓猄toreFile瀛樺湪銆侻ajor Compaction鍙互鎵嬪姩鎴栬嚜鍔ㄨЕ鍙戯紝鐒惰岀敱浜庡畠浼氬紩璧峰緢澶氱殑IO鎿嶄綔鑰屽紩璧锋ц兘闂锛屽洜鑰屽畠涓鑸細琚畨鎺掑湪鍛ㄦ湯銆佸噷鏅ㄧ瓑闆嗙兢姣旇緝闂茬殑鏃堕棿銆?br /></li></ol>鏇村舰璞′竴鐐癸紝濡備笅闈袱寮犲浘鍒嗗埆琛ㄧずMinor Compaction鍜孧ajor Compaction銆?br /><img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig18.png" height="329" width="723" /><img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig19.png" height="339" width="653" /><br /><h2 class="vpdh">HRegion Split</h2>鏈鍒濓紝涓涓猅able鍙湁涓涓狧Region锛岄殢鐫鏁版嵁鍐欏叆澧炲姞锛屽鏋滀竴涓狧Region鍒拌揪涓瀹氱殑澶у皬锛屽氨闇瑕丼plit鎴愪袱涓狧Region锛岃繖涓ぇ灏忕敱hbase.hregion.max.filesize鎸囧畾锛岄粯璁や负10GB銆傚綋split鏃讹紝涓や釜鏂扮殑HRegion浼氬湪鍚屼竴涓狧RegionServer涓垱寤猴紝瀹冧滑鍚勮嚜鍖呭惈鐖禜Region涓鍗婄殑鏁版嵁锛屽綋Split瀹屾垚鍚庯紝鐖禜Region浼氫笅绾匡紝鑰屾柊鐨勪袱涓瓙HRegion浼氬悜HMaster娉ㄥ唽涓婄嚎锛屽浜庤礋杞藉潎琛$殑鑰冭檻锛岃繖涓や釜鏂扮殑HRegion鍙兘浼氳HMaster鍒嗛厤鍒板叾浠栫殑HRegionServer涓傚叧浜嶴plit鐨勮缁嗕俊鎭紝鍙互鍙傝冭繖绡囨枃绔狅細<a >銆夾pache HBase Region Splitting and Merging銆?/a>銆?br /><img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig21.png" height="361" width="675" /><br /><h2 class="vpdh">HRegion璐熻浇鍧囪 </h2>鍦℉Region Split鍚庯紝涓や釜鏂扮殑HRegion鏈鍒濅細鍜屼箣鍓嶇殑鐖禜Region鍦ㄧ浉鍚岀殑HRegionServer涓婏紝鍑轰簬璐熻浇鍧囪 鐨勮冭檻锛孒Master鍙兘浼氬皢鍏朵腑鐨勪竴涓敋鑷充袱涓噸鏂板垎閰嶇殑鍏朵粬鐨凥RegionServer涓紝姝ゆ椂浼氬紩璧锋湁浜汬RegionServer澶勭悊鐨勬暟鎹湪鍏朵粬鑺傜偣涓婏紝鐩村埌涓嬩竴娆ajor Compaction灏嗘暟鎹粠杩滅鐨勮妭鐐圭Щ鍔ㄥ埌鏈湴鑺傜偣銆?br /><br /><img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig22.png" height="358" width="714" /><br /><h2 class="vpdh">HRegionServer Recovery</h2>褰撲竴鍙癏RegionServer瀹曟満鏃讹紝鐢变簬瀹冧笉鍐嶅彂閫丠eartbeat缁橺ooKeeper鑰岃鐩戞祴鍒帮紝姝ゆ椂ZooKeeper浼氶氱煡HMaster锛孒Master浼氭娴嬪埌鍝彴HRegionServer瀹曟満锛屽畠灏嗗畷鏈虹殑HRegionServer涓殑HRegion閲嶆柊鍒嗛厤缁欏叾浠栫殑HRegionServer锛屽悓鏃禜Master浼氭妸瀹曟満鐨凥RegionServer鐩稿叧鐨刉AL鎷嗗垎鍒嗛厤缁欑浉搴旂殑HRegionServer(灏嗘媶鍒嗗嚭鐨刉AL鏂囦欢鍐欏叆瀵瑰簲鐨勭洰鐨凥RegionServer鐨刉AL鐩綍涓紝骞跺苟鍐欏叆瀵瑰簲鐨凞ataNode涓級锛屼粠鑰岃繖浜汬RegionServer鍙互Replay鍒嗗埌鐨刉AL鏉ラ噸寤篗emStore銆?br /><img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig25.png" height="368" width="708" /><br /><img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig26.png" height="378" width="724" /><br /><h2 class="vpdh">HBase鏋舵瀯绠鍗曟荤粨</h2>鍦∟oSQL涓紝瀛樺湪钁楀悕鐨凜AP鐞嗚锛屽嵆Consistency銆丄vailability銆丳artition Tolerance涓嶅彲鍏ㄥ緱锛岀洰鍓嶅競鍦轰笂鍩烘湰涓婄殑NoSQL閮介噰鐢≒artition Tolerance浠ュ疄鐜版暟鎹緱姘村钩鎵╁睍锛屾潵澶勭悊Relational DataBase閬囧埌鐨勬棤娉曞鐞嗘暟鎹噺澶ぇ鐨勯棶棰橈紝鎴栧紩璧风殑鎬ц兘闂銆傚洜鑰屽彧鏈夊墿涓婥鍜孉鍙互閫夋嫨銆侶Base鍦ㄤ袱鑰呬箣闂撮夋嫨浜咰onsistency锛岀劧鍚庝娇鐢ㄥ涓狧Master浠ュ強鏀寔HRegionServer鐨刦ailure鐩戞帶銆乑ooKeeper寮曞叆浣滀负鍗忚皟鑰呯瓑鍚勭鎵嬫鏉ヨВ鍐矨vailability闂锛岀劧鑰屽綋缃戠粶鐨凷plit-Brain(Network Partition)鍙戠敓鏃讹紝瀹冭繕鏄棤娉曞畬鍏ㄨВ鍐矨vailability鐨勯棶棰樸備粠杩欎釜瑙掑害涓婏紝Cassandra閫夋嫨浜咥锛屽嵆瀹冨湪缃戠粶Split-Brain鏃惰繕鏄兘姝e父鍐欙紝鑰屼娇鐢ㄥ叾浠栨妧鏈潵瑙e喅Consistency鐨勯棶棰橈紝濡傝鐨勬椂鍊欒Е鍙慍onsistency鍒ゆ柇鍜屽鐞嗐傝繖鏄璁′笂鐨勯檺鍒躲?br /><br />浠庡疄鐜颁笂鐨勪紭鐐癸細<br /><ol><li class="vpdh">HBase閲囩敤寮轰竴鑷存фā鍨嬶紝鍦ㄤ竴涓啓杩斿洖鍚庯紝淇濊瘉鎵鏈夌殑璇婚兘璇诲埌鐩稿悓鐨勬暟鎹?/li><li class="vpdh">閫氳繃HRegion鍔ㄦ丼plit鍜孧erge瀹炵幇鑷姩鎵╁睍锛屽苟浣跨敤HDFS鎻愪緵鐨勫涓暟鎹浠藉姛鑳斤紝瀹炵幇楂樺彲鐢ㄦс?/li><li class="vpdh">閲囩敤HRegionServer鍜孌ataNode杩愯鍦ㄧ浉鍚岀殑鏈嶅姟鍣ㄤ笂瀹炵幇鏁版嵁鐨勬湰鍦板寲锛屾彁鍗囪鍐欐ц兘锛屽苟鍑忓皯缃戠粶鍘嬪姏銆?/li><li class="vpdh">鍐呭缓HRegionServer鐨勫畷鏈鸿嚜鍔ㄦ仮澶嶃傞噰鐢╓AL鏉eplay杩樻湭鎸佷箙鍖栧埌HDFS鐨勬暟鎹?/li><li class="vpdh">鍙互鏃犵紳鐨勫拰Hadoop/MapReduce闆嗘垚銆?br /></li></ol>瀹炵幇涓婄殑缂虹偣锛?br /><ol><li class="vpdh">WAL鐨凴eplay杩囩▼鍙兘浼氬緢鎱€?/li><li class="vpdh">鐏鹃毦鎭㈠姣旇緝澶嶆潅锛屼篃浼氭瘮杈冩參銆?/li><li class="vpdh">Major Compaction浼氬紩璧稩O Storm銆?/li><li class="vpdh">銆傘傘傘?br /></li></ol><h2 class="vpdh">鍙傝冿細</h2> https://www.mapr.com/blog/in-depth-look-hbase-architecture#.VdNSN6Yp3qx<br /> //jimbojw.com/wiki/index.php?title=Understanding_Hbase_and_BigTable<br /> //hbase.apache.org/book.html <br /> //www.searchtb.com/2011/01/understanding-hbase.html <br /> //research.google.com/archive/bigtable-osdi06.pdf<img src ="//www.cqk4s.cn/DLevin/aggbug/426950.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="//www.cqk4s.cn/DLevin/" target="_blank">DLevin</a> 2015-08-22 19:40 <a href="//www.cqk4s.cn/DLevin/archive/2015/08/22/426950.html#Feedback" target="_blank" style="text-decoration:none;">鍙戣〃璇勮</a></div>]]></description></item><item><title>娣卞叆HBase鏋舵瀯瑙f瀽锛堜竴锛?/title><link>//www.cqk4s.cn/DLevin/archive/2015/08/22/426877.html</link><dc:creator>DLevin</dc:creator><author>DLevin</author><pubDate>Sat, 22 Aug 2015 09:44:00 GMT</pubDate><guid>//www.cqk4s.cn/DLevin/archive/2015/08/22/426877.html</guid><wfw:comment>//www.cqk4s.cn/DLevin/comments/426877.html</wfw:comment><comments>//www.cqk4s.cn/DLevin/archive/2015/08/22/426877.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>//www.cqk4s.cn/DLevin/comments/commentRss/426877.html</wfw:commentRss><trackback:ping>//www.cqk4s.cn/DLevin/services/trackbacks/426877.html</trackback:ping><description><![CDATA[<h2 class="vpdh">鍓嶈</h2> 鍏徃鍐呴儴浣跨敤鐨勬槸MapR鐗堟湰鐨凥adoop鐢熸佺郴缁燂紝鍥犺屼粠MapR鐨勫畼缃戠湅鍒颁簡杩欑瘒鏂囨枃绔狅細<a >An In-Depth Look at the HBase Architecture</a>锛屽師鏈兂缈昏瘧鍏ㄦ枃锛岀劧鑰屽鏋滅炕璇戝氨闇瑕佸悇绉嶅挰鏂囧毤瀛楋紝澶夯鐑︼紝鍥犺屾湰鏂囧ぇ閮ㄥ垎浣跨敤浜嗚嚜宸辩殑璇█锛屽苟涓斿姞鍏ヤ簡鍏朵粬璧勬簮鐨勫弬鑰冪悊瑙d互鍙婃湰浜鸿嚜宸辫婧愮爜鏃跺鍏剁殑鐞嗚В锛屽睘浜庡崐缈昏瘧銆佸崐鍘熷垱鍚с?br /> <h2 class="vpdh">HBase鏋舵瀯缁勬垚</h2> HBase閲囩敤Master/Slave鏋舵瀯鎼缓闆嗙兢锛屽畠闅跺睘浜嶩adoop鐢熸佺郴缁燂紝鐢变竴涓嬬被鍨嬭妭鐐圭粍鎴愶細HMaster鑺傜偣銆丠RegionServer鑺傜偣銆乑ooKeeper闆嗙兢锛岃屽湪搴曞眰锛屽畠灏嗘暟鎹瓨鍌ㄤ簬HDFS涓紝鍥犺屾秹鍙婂埌HDFS鐨凬ameNode銆丏ataNode绛夛紝鎬讳綋缁撴瀯濡備笅锛?br /> <img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArch1.jpg" height="389" width="603" /><br /> 鍏朵腑<strong class="vpdh">HMaster鑺傜偣</strong>鐢ㄤ簬锛?br /> <ol> <li class="vpdh">绠$悊HRegionServer锛屽疄鐜板叾璐熻浇鍧囪 銆?/li> <li class="vpdh">绠$悊鍜屽垎閰岺Region锛屾瘮濡傚湪HRegion split鏃跺垎閰嶆柊鐨凥Region锛涘湪HRegionServer閫鍑烘椂杩佺Щ鍏跺唴鐨凥Region鍒板叾浠朒RegionServer涓娿?/li> <li class="vpdh">瀹炵幇DDL鎿嶄綔锛圖ata Definition Language锛宯amespace鍜宼able鐨勫鍒犳敼锛宑olumn familiy鐨勫鍒犳敼绛夛級銆?/li> <li class="vpdh">绠$悊namespace鍜宼able鐨勫厓鏁版嵁锛堝疄闄呭瓨鍌ㄥ湪HDFS涓婏級銆?/li> <li class="vpdh">鏉冮檺鎺у埗锛圓CL锛夈?/li> </ol> <strong class="vpdh">HRegionServer鑺傜偣</strong>鐢ㄤ簬锛?br /> <ol> <li class="vpdh">瀛樻斁鍜岀鐞嗘湰鍦癏Region銆?/li> <li class="vpdh">璇诲啓HDFS锛岀鐞員able涓殑鏁版嵁銆?/li> <li class="vpdh">Client鐩存帴閫氳繃HRegionServer璇诲啓鏁版嵁锛堜粠HMaster涓幏鍙栧厓鏁版嵁锛屾壘鍒癛owKey鎵鍦ㄧ殑HRegion/HRegionServer鍚庯級銆?/li> </ol> <strong class="vpdh">ZooKeeper闆嗙兢鏄崗璋冪郴缁?/strong>锛岀敤浜庯細<br /> <ol> <li class="vpdh">瀛樻斁鏁翠釜 HBase闆嗙兢鐨勫厓鏁版嵁浠ュ強闆嗙兢鐨勭姸鎬佷俊鎭?/li> <li class="vpdh">瀹炵幇HMaster涓讳粠鑺傜偣鐨刦ailover銆?/li> </ol> HBase Client閫氳繃RPC鏂瑰紡鍜孒Master銆丠RegionServer閫氫俊锛涗竴涓狧RegionServer鍙互瀛樻斁1000涓狧Region锛涘簳灞俆able鏁版嵁瀛樺偍浜嶩DFS涓紝鑰孒Region鎵澶勭悊鐨勬暟鎹敖閲忓拰鏁版嵁鎵鍦ㄧ殑DataNode鍦ㄤ竴璧凤紝瀹炵幇鏁版嵁鐨勬湰鍦板寲锛涙暟鎹湰鍦板寲骞朵笉鏄昏兘瀹炵幇锛屾瘮濡傚湪HRegion绉诲姩(濡傚洜Split)鏃讹紝闇瑕佺瓑涓嬩竴娆ompact鎵嶈兘缁х画鍥炲埌鏈湴鍖栥?br /> <br /> 鏈潃鍗婄炕璇戠殑鍘熷垯锛屽啀璐翠竴涓夾n In-Depth Look At The HBase Architecture銆嬬殑鏋舵瀯鍥撅細<br /> <img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig1.png" height="343" width="632" /><br /> 杩欎釜鏋舵瀯鍥炬瘮杈冩竻鏅扮殑琛ㄨ揪浜咹Master鍜孨ameNode閮芥敮鎸佸涓儹澶囦唤锛屼娇鐢╖ooKeeper鏉ュ仛鍗忚皟锛沍ooKeeper骞朵笉鏄簯鑸绉橈紝瀹冧竴鑸敱涓夊彴鏈哄櫒缁勬垚涓涓泦缇わ紝鍐呴儴浣跨敤PAXOS绠楁硶鏀寔涓夊彴Server涓殑涓鍙板畷鏈猴紝涔熸湁浣跨敤浜斿彴鏈哄櫒鐨勶紝姝ゆ椂鍒欏彲浠ユ敮鎸佸悓鏃朵袱鍙板畷鏈猴紝鏃㈠皯浜庡崐鏁扮殑瀹曟満锛岀劧鑰岄殢鐫鏈哄櫒鐨勫鍔狅紝瀹冪殑鎬ц兘涔熶細涓嬮檷锛汻egionServer鍜孌ataNode涓鑸細鏀惧湪鐩稿悓鐨凷erver涓婂疄鐜版暟鎹殑鏈湴鍖栥?br /> <h2 class="vpdh">HRegion</h2> HBase浣跨敤RowKey灏嗚〃姘村钩鍒囧壊鎴愬涓狧Region锛屼粠HMaster鐨勮搴︼紝姣忎釜HRegion閮界邯褰曚簡瀹冪殑StartKey鍜孍ndKey锛堢涓涓狧Region鐨凷tartKey涓虹┖锛屾渶鍚庝竴涓狧Region鐨凟ndKey涓虹┖锛夛紝鐢变簬RowKey鏄帓搴忕殑锛屽洜鑰孋lient鍙互閫氳繃HMaster蹇熺殑瀹氫綅姣忎釜RowKey鍦ㄥ摢涓狧Region涓侶Region鐢盚Master鍒嗛厤鍒扮浉搴旂殑HRegionServer涓紝鐒跺悗鐢盚RegionServer璐熻矗HRegion鐨勫惎鍔ㄥ拰绠$悊锛屽拰Client鐨勯氫俊锛岃礋璐f暟鎹殑璇?浣跨敤HDFS)銆傛瘡涓狧RegionServer鍙互鍚屾椂绠$悊1000涓乏鍙崇殑HRegion锛堣繖涓暟瀛楁庝箞鏉ョ殑锛熸病鏈変粠浠g爜涓湅鍒伴檺鍒讹紝闅鹃亾鏄嚭浜庣粡楠岋紵瓒呰繃1000涓細寮曡捣鎬ц兘闂锛?strong>鏉ュ洖绛旇繖涓棶棰?/strong>锛氭劅瑙夎繖涓?000鐨勬暟瀛楁槸浠嶣igTable鐨勮鏂囦腑鏉ョ殑锛? Implementation鑺傦級锛欵ach tablet server manages a set of tablets(typically we have somewhere between ten to a thousand tablets per tablet server)锛夈?br /> <img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig2.png" height="337" width="724" /><br /> <h2 class="vpdh">HMaster</h2> HMaster娌℃湁鍗曠偣鏁呴殰闂锛屽彲浠ュ惎鍔ㄥ涓狧Master锛岄氳繃ZooKeeper鐨凪aster Election鏈哄埗淇濊瘉鍚屾椂鍙湁涓涓狧Master鍑轰簬Active鐘舵侊紝鍏朵粬鐨凥Master鍒欏浜庣儹澶囦唤鐘舵併備竴鑸儏鍐典笅浼氬惎鍔ㄤ袱涓狧Master锛岄潪Active鐨凥Master浼氬畾鏈熺殑鍜孉ctive HMaster閫氫俊浠ヨ幏鍙栧叾鏈鏂扮姸鎬侊紝浠庤屼繚璇佸畠鏄疄鏃舵洿鏂扮殑锛屽洜鑰屽鏋滃惎鍔ㄤ簡澶氫釜HMaster鍙嶈屽鍔犱簡Active HMaster鐨勮礋鎷呫傚墠鏂囧凡缁忎粙缁嶈繃浜咹Master鐨勪富瑕佺敤浜嶩Region鐨勫垎閰嶅拰绠$悊锛孌DL(Data Definition Language锛屾棦Table鐨勬柊寤恒佸垹闄ゃ佷慨鏀圭瓑)鐨勫疄鐜扮瓑锛屾棦瀹冧富瑕佹湁涓ゆ柟闈㈢殑鑱岃矗锛?br /> <ol> <li class="vpdh">鍗忚皟HRegionServer <ol> <li class="vpdh">鍚姩鏃禜Region鐨勫垎閰嶏紝浠ュ強璐熻浇鍧囪 鍜屼慨澶嶆椂HRegion鐨勯噸鏂板垎閰嶃?/li> <li class="vpdh">鐩戞帶闆嗙兢涓墍鏈塇RegionServer鐨勭姸鎬?閫氳繃Heartbeat鍜岀洃鍚琙ooKeeper涓殑鐘舵?銆?br /> </li> </ol> </li> <li class="vpdh">Admin鑱岃兘 <ol> <li class="vpdh">鍒涘缓銆佸垹闄ゃ佷慨鏀筎able鐨勫畾涔夈?br /> </li> </ol> </li> </ol> <img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig3.png" /><br /> <h2 class="vpdh"> ZooKeeper锛氬崗璋冭?/h2> ZooKeeper涓篐Base闆嗙兢鎻愪緵鍗忚皟鏈嶅姟锛屽畠绠$悊鐫HMaster鍜孒RegionServer鐨勭姸鎬?available/alive绛?锛屽苟涓斾細鍦ㄥ畠浠畷鏈烘椂閫氱煡缁橦Master锛屼粠鑰孒Master鍙互瀹炵幇HMaster涔嬮棿鐨刦ailover锛屾垨瀵瑰畷鏈虹殑HRegionServer涓殑HRegion闆嗗悎鐨勪慨澶?灏嗗畠浠垎閰嶇粰鍏朵粬鐨凥RegionServer)銆俍ooKeeper闆嗙兢鏈韩浣跨敤涓鑷存у崗璁?PAXOS鍗忚)淇濊瘉姣忎釜鑺傜偣鐘舵佺殑涓鑷存с?br /> <img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig4.png" height="318" width="703" /><br /> <h2 class="vpdh">How The Components Work Together</h2> ZooKeeper鍗忚皟闆嗙兢鎵鏈夎妭鐐圭殑鍏变韩淇℃伅锛屽湪HMaster鍜孒RegionServer杩炴帴鍒癦ooKeeper鍚庡垱寤篍phemeral鑺傜偣锛屽苟浣跨敤Heartbeat鏈哄埗缁存寔杩欎釜鑺傜偣鐨勫瓨娲荤姸鎬侊紝濡傛灉鏌愪釜Ephemeral鑺傜偣瀹炴晥锛屽垯HMaster浼氭敹鍒伴氱煡锛屽苟鍋氱浉搴旂殑澶勭悊銆?br /> <img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig5.png" height="329" width="722" /><br /> 鍙﹀锛孒Master閫氳繃鐩戝惉ZooKeeper涓殑Ephemeral鑺傜偣(榛樿锛?hbase/rs/*)鏉ョ洃鎺RegionServer鐨勫姞鍏ュ拰瀹曟満銆傚湪绗竴涓狧Master杩炴帴鍒癦ooKeeper鏃朵細鍒涘缓Ephemeral鑺傜偣(榛樿锛?hbasae/master)鏉ヨ〃绀篈ctive鐨凥Master锛屽叾鍚庡姞杩涙潵鐨凥Master鍒欑洃鍚Ephemeral鑺傜偣锛屽鏋滃綋鍓岮ctive鐨凥Master瀹曟満锛屽垯璇ヨ妭鐐规秷澶憋紝鍥犺屽叾浠朒Master寰楀埌閫氱煡锛岃屽皢鑷韩杞崲鎴怉ctive鐨凥Master锛屽湪鍙樹负Active鐨凥Master涔嬪墠锛屽畠浼氬垱寤哄湪/hbase/back-masters/涓嬪垱寤鸿嚜宸辩殑Ephemeral鑺傜偣銆?br /> <h3> HBase鐨勭涓娆¤鍐?/h3> 鍦℉Base 0.96浠ュ墠锛孒Base鏈変袱涓壒娈婄殑Table锛?ROOT-鍜?META.锛堝<a >BigTable</a>涓殑璁捐锛夛紝鍏朵腑-ROOT- Table鐨勪綅缃瓨鍌ㄥ湪ZooKeeper锛屽畠瀛樺偍浜?META. Table鐨凴egionInfo淇℃伅锛屽苟涓斿畠鍙兘瀛樺湪涓涓狧Region锛岃?META. Table鍒欏瓨鍌ㄤ簡鐢ㄦ埛Table鐨凴egionInfo淇℃伅锛屽畠鍙互琚垏鍒嗘垚澶氫釜HRegion锛屽洜鑰屽绗竴娆¤闂敤鎴稵able鏃讹紝棣栧厛浠嶼ooKeeper涓鍙?ROOT- Table鎵鍦℉RegionServer锛涚劧鍚庝粠璇RegionServer涓牴鎹姹傜殑TableName锛孯owKey璇诲彇.META. Table鎵鍦℉RegionServer锛涙渶鍚庝粠璇RegionServer涓鍙?META. Table鐨勫唴瀹硅岃幏鍙栨娆¤姹傞渶瑕佽闂殑HRegion鎵鍦ㄧ殑浣嶇疆锛岀劧鍚庤闂HRegionSever鑾峰彇璇锋眰鐨勬暟鎹紝杩欓渶瑕佷笁娆¤姹傛墠鑳芥壘鍒扮敤鎴稵able鎵鍦ㄧ殑浣嶇疆锛岀劧鍚庣鍥涙璇锋眰寮濮嬭幏鍙栫湡姝g殑鏁版嵁銆傚綋鐒朵负浜嗘彁鍗囨ц兘锛屽鎴风浼氱紦瀛?ROOT- Table浣嶇疆浠ュ強-ROOT-/.META. Table鐨勫唴瀹广傚涓嬪浘鎵绀猴細<br /> <img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/image0030.jpg" height="228" width="399" /><br /> 鍙槸鍗充娇瀹㈡埛绔湁缂撳瓨锛屽湪鍒濆闃舵闇瑕佷笁娆¤姹傛墠鑳界洿鍒扮敤鎴稵able鐪熸鎵鍦ㄧ殑浣嶇疆涔熸槸鎬ц兘浣庝笅鐨勶紝鑰屼笖鐪熺殑鏈夊繀瑕佹敮鎸侀偅涔堝鐨凥Region鍚楋紵鎴栬瀵笹oogle杩欐牱鐨勫叕鍙告潵璇存槸闇瑕佺殑锛屼絾鏄涓鑸殑闆嗙兢鏉ヨ濂藉儚骞舵病鏈夎繖涓繀瑕併傚湪BigTable鐨勮鏂囦腑璇达紝姣忚METADATA瀛樺偍1KB宸﹀彸鏁版嵁锛屼腑绛夊ぇ灏忕殑Tablet(HRegion)鍦?28MB宸﹀彸锛?灞備綅缃殑Schema璁捐鍙互鏀寔2^34涓猅ablet(HRegion)銆傚嵆浣垮幓鎺?ROOT- Table锛屼篃杩樺彲浠ユ敮鎸?^17(131072)涓狧Region锛?濡傛灉姣忎釜HRegion杩樻槸128MB锛岄偅灏辨槸16TB锛岃繖涓矊浼间笉澶熷ぇ锛屼絾鏄幇鍦ㄧ殑HRegion鐨勬渶澶уぇ灏忛兘浼氳缃殑姣旇緝澶э紝姣斿鎴戜滑璁剧疆浜?GB锛屾鏃舵敮鎸佺殑澶у皬鍒欏彉鎴愪簡4PB锛屽涓鑸殑闆嗙兢鏉ヨ宸茬粡澶熶簡锛屽洜鑰屽湪HBase 0.96浠ュ悗鍘绘帀浜?ROOT- Table锛屽彧鍓╀笅杩欎釜鐗规畩鐨勭洰褰曡〃鍙仛Meta Table(hbase:meta)锛屽畠瀛樺偍浜嗛泦缇や腑鎵鏈夌敤鎴稨Region鐨勪綅缃俊鎭紝鑰孼ooKeeper鐨勮妭鐐逛腑(/hbase/meta-region-server)瀛樺偍鐨勫垯鐩存帴鏄繖涓狹eta Table鐨勪綅缃紝骞朵笖杩欎釜Meta Table濡備互鍓嶇殑-ROOT- Table涓鏍锋槸涓嶅彲split鐨勩傝繖鏍凤紝瀹㈡埛绔湪绗竴娆¤闂敤鎴稵able鐨勬祦绋嬪氨鍙樻垚浜嗭細<br /> <ol> <li class="vpdh">浠嶼ooKeeper(/hbase/meta-region-server)涓幏鍙杊base:meta鐨勪綅缃紙HRegionServer鐨勪綅缃級锛岀紦瀛樿浣嶇疆淇℃伅銆?/li> <li class="vpdh">浠嶩RegionServer涓煡璇㈢敤鎴稵able瀵瑰簲璇锋眰鐨凴owKey鎵鍦ㄧ殑HRegionServer锛岀紦瀛樿浣嶇疆淇℃伅銆?/li> <li class="vpdh">浠庢煡璇㈠埌HRegionServer涓鍙朢ow銆?/li> </ol> 浠庤繖涓繃绋嬩腑锛屾垜浠彂鐜板鎴蜂細缂撳瓨杩欎簺浣嶇疆淇℃伅锛岀劧鑰岀浜屾瀹冨彧鏄紦瀛樺綋鍓峈owKey瀵瑰簲鐨凥Region鐨勪綅缃紝鍥犺屽鏋滀笅涓涓鏌ョ殑RowKey涓嶅湪鍚屼竴涓狧Region涓紝鍒欓渶瑕佺户缁煡璇base:meta鎵鍦ㄧ殑HRegion锛岀劧鑰岄殢鐫鏃堕棿鐨勬帹绉伙紝瀹㈡埛绔紦瀛樼殑浣嶇疆淇℃伅瓒婃潵瓒婂锛屼互鑷充簬涓嶉渶瑕佸啀娆℃煡鎵緃base:meta Table鐨勪俊鎭紝闄ら潪鏌愪釜HRegion鍥犱负瀹曟満鎴朣plit琚Щ鍔紝姝ゆ椂闇瑕侀噸鏂版煡璇㈠苟涓旀洿鏂扮紦瀛樸?br /> <img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig6.png" height="356" width="590" /><br /> <h3> hbase:meta琛?/h3> hbase:meta琛ㄥ瓨鍌ㄤ簡鎵鏈夌敤鎴稨Region鐨勪綅缃俊鎭紝瀹冪殑RowKey鏄細tableName,regionStartKey,regionId,replicaId绛夛紝瀹冨彧鏈塱nfo鍒楁棌锛岃繖涓垪鏃忓寘鍚笁涓垪锛屼粬浠垎鍒槸锛歩nfo:regioninfo鍒楁槸RegionInfo鐨刾roto鏍煎紡锛歳egionId,tableName,startKey,endKey,offline,split,replicaId锛沬nfo:server鏍煎紡锛欻RegionServer瀵瑰簲鐨剆erver:port锛沬nfo:serverstartcode鏍煎紡鏄疕RegionServer鐨勫惎鍔ㄦ椂闂存埑銆?br /> <img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig7.png" height="362" width="736" /><br /> <h2 class="vpdh">HRegionServer璇﹁В</h2> HRegionServer涓鑸拰DataNode鍦ㄥ悓涓鍙版満鍣ㄤ笂杩愯锛屽疄鐜版暟鎹殑鏈湴鎬с侶RegionServer鍖呭惈澶氫釜HRegion锛岀敱WAL(HLog)銆丅lockCache銆丮emStore銆丠File缁勬垚銆?br /> <ol> <li class="vpdh"><strong class="vpdh">WAL鍗砏rite Ahead Log</strong>锛屽湪鏃╂湡鐗堟湰涓О涓篐Log锛屽畠鏄疕DFS涓婄殑涓涓枃浠讹紝濡傚叾鍚嶅瓧鎵琛ㄧず鐨勶紝鎵鏈夊啓鎿嶄綔閮戒細鍏堜繚璇佸皢鏁版嵁鍐欏叆杩欎釜Log鏂囦欢鍚庯紝鎵嶄細鐪熸鏇存柊MemStore锛屾渶鍚庡啓鍏File涓傞噰鐢ㄨ繖绉嶆ā寮忥紝鍙互淇濊瘉HRegionServer瀹曟満鍚庯紝鎴戜滑渚濈劧鍙互浠庤Log鏂囦欢涓鍙栨暟鎹紝Replay鎵鏈夌殑鎿嶄綔锛岃屼笉鑷充簬鏁版嵁涓㈠け銆傝繖涓狶og鏂囦欢浼氬畾鏈烺oll鍑烘柊鐨勬枃浠惰屽垹闄ゆ棫鐨勬枃浠?閭d簺宸叉寔涔呭寲鍒癏File涓殑Log鍙互鍒犻櫎)銆俉AL鏂囦欢瀛樺偍鍦?hbase/WALs/${HRegionServer_Name}鐨勭洰褰曚腑(鍦?.94涔嬪墠锛屽瓨鍌ㄥ湪/hbase/.logs/鐩綍涓?锛屼竴鑸竴涓狧RegionServer鍙湁涓涓猈AL瀹炰緥锛屼篃灏辨槸璇翠竴涓狧RegionServer鐨勬墍鏈塛AL鍐欓兘鏄覆琛岀殑(灏卞儚log4j鐨勬棩蹇楀啓涔熸槸涓茶鐨?锛岃繖褰撶劧浼氬紩璧锋ц兘闂锛屽洜鑰屽湪HBase 1.0涔嬪悗锛岄氳繃<a >HBASE-5699</a>瀹炵幇浜嗗涓猈AL骞惰鍐?MultiWAL)锛岃瀹炵幇閲囩敤HDFS鐨勫涓閬撳啓锛屼互鍗曚釜HRegion涓哄崟浣嶃傚叧浜嶹AL鍙互鍙傝僕ikipedia鐨?a >Write-Ahead Logging</a>銆傞『渚垮悙妲戒竴鍙ワ紝鑻辨枃鐗堢殑缁村熀鐧剧绔熺劧鑳芥鏃犲帇鍔涚殑姝e父璁块棶浜嗭紝杩欐槸鏌愪釜GFW鐨勭枏蹇借繕鏄互鍚庣殑甯告侊紵</li> <li class="vpdh"><strong class="vpdh">BlockCache鏄竴涓缂撳瓨</strong>锛屽嵆“寮曠敤灞閮ㄦ?#8221;鍘熺悊锛堜篃搴旂敤浜嶤PU锛?a >鍒嗙┖闂村眬閮ㄦу拰鏃堕棿灞閮ㄦ?/a>锛岀┖闂村眬閮ㄦф槸鎸嘋PU鍦ㄦ煇涓鏃跺埢闇瑕佹煇涓暟鎹紝閭d箞鏈夊緢澶х殑姒傜巼鍦ㄤ竴涓嬫椂鍒诲畠闇瑕佺殑鏁版嵁鍦ㄥ叾闄勮繎锛涙椂闂村眬閮ㄦф槸鎸囨煇涓暟鎹湪琚闂繃涓娆″悗锛屽畠鏈夊緢澶х殑姒傜巼鍦ㄤ笉涔呯殑灏嗘潵浼氳鍐嶆鐨勮闂級锛屽皢鏁版嵁棰勮鍙栧埌鍐呭瓨涓紝浠ユ彁鍗囪鐨勬ц兘銆侶Base涓彁渚涗袱绉岯lockCache鐨勫疄鐜帮細榛樿on-heap LruBlockCache鍜孊ucketCache(閫氬父鏄痮ff-heap)銆傞氬父BucketCache鐨勬ц兘瑕佸樊浜嶭ruBlockCache锛岀劧鑰岀敱浜嶨C鐨勫奖鍝嶏紝LruBlockCache鐨勫欢杩熶細鍙樼殑涓嶇ǔ瀹氾紝鑰孊ucketCache鐢变簬鏄嚜宸辩鐞咮lockCache锛岃屼笉闇瑕丟C锛屽洜鑰屽畠鐨勫欢杩熼氬父姣旇緝绋冲畾锛岃繖涔熸槸鏈変簺鏃跺欓渶瑕侀夌敤BucketCache鐨勫師鍥犮傝繖绡囨枃绔?a >BlockCache101</a>瀵筼n-heap鍜宱ff-heap鐨凚lockCache鍋氫簡璇︾粏鐨勬瘮杈冦?/li><strong class="vpdh"> </strong><li class="vpdh"><strong class="vpdh">HRegion鏄竴涓猅able涓殑涓涓猂egion鍦ㄤ竴涓狧RegionServer涓殑琛ㄨ揪</strong>銆備竴涓猅able鍙互鏈変竴涓垨澶氫釜Region锛屼粬浠彲浠ュ湪涓涓浉鍚岀殑HRegionServer涓婏紝涔熷彲浠ュ垎甯冨湪涓嶅悓鐨凥RegionServer涓婏紝涓涓狧RegionServer鍙互鏈夊涓狧Region锛屼粬浠垎鍒睘浜庝笉鍚岀殑Table銆侶Region鐢卞涓猄tore(HStore)鏋勬垚锛屾瘡涓狧Store瀵瑰簲浜嗕竴涓猅able鍦ㄨ繖涓狧Region涓殑涓涓狢olumn Family锛屽嵆姣忎釜Column Family灏辨槸涓涓泦涓殑瀛樺偍鍗曞厓锛屽洜鑰屾渶濂藉皢鍏锋湁鐩歌繎IO鐗规х殑Column瀛樺偍鍦ㄤ竴涓狢olumn Family锛屼互瀹炵幇楂樻晥璇诲彇(鏁版嵁灞閮ㄦу師鐞嗭紝鍙互鎻愰珮缂撳瓨鐨勫懡涓巼)銆侶Store鏄疕Base涓瓨鍌ㄧ殑鏍稿績锛屽畠瀹炵幇浜嗚鍐橦DFS鍔熻兘锛屼竴涓狧Store鐢变竴涓狹emStore 鍜?涓垨澶氫釜StoreFile缁勬垚銆?br /> <ol> <li class="vpdh"><strong class="vpdh">MemStore鏄竴涓啓缂撳瓨</strong>(In Memory Sorted Buffer)锛屾墍鏈夋暟鎹殑鍐欏湪瀹屾垚WAL鏃ュ織鍐欏悗锛屼細 鍐欏叆MemStore涓紝鐢盡emStore鏍规嵁涓瀹氱殑绠楁硶灏嗘暟鎹瓼lush鍒板湴灞侶DFS鏂囦欢涓?HFile)锛岄氬父姣忎釜HRegion涓殑姣忎釜 Column Family鏈変竴涓嚜宸辩殑MemStore銆?/li> <li class="vpdh"><strong class="vpdh">HFile(StoreFile) 鐢ㄤ簬瀛樺偍HBase鐨勬暟鎹?Cell/KeyValue)</strong>銆傚湪HFile涓殑鏁版嵁鏄寜RowKey銆丆olumn Family銆丆olumn鎺掑簭锛屽鐩稿悓鐨凜ell(鍗宠繖涓変釜鍊奸兘涓鏍?锛屽垯鎸塼imestamp鍊掑簭鎺掑垪銆?/li> </ol> </li> </ol> <img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig8.png" /><br /> 铏界劧涓婇潰杩欏紶鍥惧睍鐜扮殑鏄渶鏂扮殑HRegionServer鐨勬灦鏋?浣嗘槸骞朵笉鏄偅涔堢殑绮剧‘)锛屼絾鏄垜涓鐩存瘮杈冨枩娆㈢湅浠ヤ笅杩欏紶鍥撅紝鍗充娇瀹冨睍鐜扮殑搴旇鏄?.94浠ュ墠鐨勬灦鏋勩?br /> <img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/image0060.jpg" height="347" width="553" /><br /> <h3> HRegionServer涓暟鎹啓娴佺▼鍥捐В</h3> 褰撳鎴风鍙戣捣涓涓狿ut璇锋眰鏃讹紝棣栧厛瀹冧粠hbase:meta琛ㄤ腑鏌ュ嚭璇ut鏁版嵁鏈缁堥渶瑕佸幓鐨凥RegionServer銆傜劧鍚庡鎴风灏哖ut璇锋眰鍙戦佺粰鐩稿簲鐨凥RegionServer锛屽湪HRegionServer涓畠棣栧厛浼氬皢璇ut鎿嶄綔鍐欏叆WAL鏃ュ織鏂囦欢涓?Flush鍒扮鐩樹腑)銆?br /><img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig9.png" height="363" width="716" /><br /> 鍐欏畬WAL鏃ュ織鏂囦欢鍚庯紝HRegionServer鏍规嵁Put涓殑TableName鍜孯owKey鎵惧埌瀵瑰簲鐨凥Region锛屽苟鏍规嵁Column Family鎵惧埌瀵瑰簲鐨凥Store锛屽苟灏哖ut鍐欏叆鍒拌HStore鐨凪emStore涓傛鏃跺啓鎴愬姛锛屽苟杩斿洖閫氱煡瀹㈡埛绔?br /><img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig10.png" height="298" width="664" /><br /><h3>MemStore Flush<br /></h3>MemStore鏄竴涓狪n Memory Sorted Buffer锛屽湪姣忎釜HStore涓兘鏈変竴涓狹emStore锛屽嵆瀹冩槸涓涓狧Region鐨勪竴涓狢olumn Family瀵瑰簲涓涓疄渚嬨傚畠鐨勬帓鍒楅『搴忎互RowKey銆丆olumn Family銆丆olumn鐨勯『搴忎互鍙奣imestamp鐨勫掑簭锛屽涓嬫墍绀猴細<br /><img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig11.png" height="351" width="719" /><br />姣忎竴娆ut/Delete璇锋眰閮芥槸鍏堝啓鍏ュ埌MemStore涓紝褰揗emStore婊″悗浼欶lush鎴愪竴涓柊鐨凷toreFile(搴曞眰瀹炵幇鏄疕File)锛屽嵆涓涓狧Store(Column Family)鍙互鏈?涓垨澶氫釜StoreFile(HFile)銆傛湁浠ヤ笅涓夌鎯呭喌鍙互瑙﹀彂MemStore鐨凢lush鍔ㄤ綔锛?strong>闇瑕佹敞鎰忕殑鏄疢emStore鐨勬渶灏廎lush鍗曞厓鏄疕Region鑰屼笉鏄崟涓狹emStore</strong>銆傛嵁璇磋繖鏄疌olumn Family鏈変釜鏁伴檺鍒剁殑鍏朵腑涓涓師鍥狅紝浼拌鏄洜涓哄お澶氱殑Column Family涓璧稦lush浼氬紩璧锋ц兘闂锛熷叿浣撳師鍥犳湁寰呰冭瘉銆?br /><ol><li class="vpdh">褰撲竴涓狧Region涓殑鎵鏈塎emStore鐨勫ぇ灏忔诲拰瓒呰繃浜唄base.hregion.memstore.flush.size鐨勫ぇ灏忥紝榛樿128MB銆傛鏃跺綋鍓嶇殑HRegion涓墍鏈夌殑MemStore浼欶lush鍒癏DFS涓?/li><li class="vpdh">褰撳叏灞MemStore鐨勫ぇ灏忚秴杩囦簡hbase.regionserver.global.memstore.upperLimit鐨勫ぇ灏忥紝榛樿40锛呯殑鍐呭瓨浣跨敤閲忋傛鏃跺綋鍓岺RegionServer涓墍鏈塇Region涓殑MemStore閮戒細Flush鍒癏DFS涓紝Flush椤哄簭鏄疢emStore澶у皬鐨勫掑簭锛堜竴涓狧Region涓墍鏈塎emStore鎬诲拰浣滀负璇Region鐨凪emStore鐨勫ぇ灏忚繕鏄夊彇鏈澶х殑MemStore浣滀负鍙傝冿紵鏈夊緟鑰冭瘉锛夛紝鐩村埌鎬讳綋鐨凪emStore浣跨敤閲忎綆浜巋base.regionserver.global.memstore.lowerLimit锛岄粯璁?8%鐨勫唴瀛樹娇鐢ㄩ噺銆?/li><li class="vpdh">褰撳墠HRegionServer涓璚AL鐨勫ぇ灏忚秴杩囦簡hbase.regionserver.hlog.blocksize * hbase.regionserver.max.logs鐨勬暟閲忥紝褰撳墠HRegionServer涓墍鏈塇Region涓殑MemStore閮戒細Flush鍒癏DFS涓紝Flush浣跨敤鏃堕棿椤哄簭锛屾渶鏃╃殑MemStore鍏團lush鐩村埌WAL鐨勬暟閲忓皯浜巋base.regionserver.hlog.blocksize * hbase.regionserver.max.logs銆?a >杩欓噷</a>璇磋繖涓や釜鐩镐箻鐨勯粯璁ゅぇ灏忔槸2GB锛屾煡浠g爜锛宧base.regionserver.max.logs榛樿鍊兼槸32锛岃宧base.regionserver.hlog.blocksize鏄疕DFS鐨勯粯璁locksize锛?2MB銆備絾涓嶇鎬庝箞鏍凤紝鍥犱负杩欎釜澶у皬瓒呰繃闄愬埗寮曡捣鐨凢lush涓嶆槸涓浠跺ソ浜嬶紝鍙兘寮曡捣闀挎椂闂寸殑寤惰繜锛屽洜鑰岃繖绡囨枃绔犵粰鐨勫缓璁細“<strong style="color: #339966; font-family: STHeiti; font-size: medium; font-style: normal; font-variant: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 1; word-spacing: 0px; -webkit-text-stroke-width: 0px;">Hint</strong><span style="color: #339966; font-family: STHeiti; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 1; word-spacing: 0px; -webkit-text-stroke-width: 0px; display: inline !important; float: none;">: keep hbase.regionserver.hlog.blocksize * hbase.regionserver.maxlogs just a bit above hbase.regionserver.global.memstore.lowerLimit * HBASE_HEAPSIZE.</span>”銆傚苟涓旈渶瑕佹敞鎰忥紝<a >杩欓噷</a>缁欑殑鎻忚堪鏄湁閿欑殑(铏界劧瀹冩槸瀹樻柟鐨勬枃妗?銆?br /></li></ol>鍦∕emStore Flush杩囩▼涓紝杩樹細鍦ㄥ熬閮ㄨ拷鍔犱竴浜沵eta鏁版嵁锛屽叾涓氨鍖呮嫭Flush鏃舵渶澶х殑WAL sequence鍊硷紝浠ュ憡璇塇Base杩欎釜StoreFile鍐欏叆鐨勬渶鏂版暟鎹殑搴忓垪锛岄偅涔堝湪Recover鏃跺氨鐩村埌浠庡摢閲屽紑濮嬨傚湪HRegion鍚姩鏃讹紝杩欎釜sequence浼氳璇诲彇锛屽苟鍙栨渶澶х殑浣滀负涓嬩竴娆℃洿鏂版椂鐨勮捣濮媠equence銆?br /><img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig12.png" height="248" width="622" /><br /><h2 class="vpdh"> HFile鏍煎紡</h2>HBase鐨勬暟鎹互KeyValue(Cell)鐨勫舰寮忛『搴忕殑瀛樺偍鍦℉File涓紝鍦∕emStore鐨凢lush杩囩▼涓敓鎴怘File锛岀敱浜嶮emStore涓瓨鍌ㄧ殑Cell閬靛惊鐩稿悓鐨勬帓鍒楅『搴忥紝鍥犺孎lush杩囩▼鏄『搴忓啓锛屾垜浠洿鍒扮鐩樼殑椤哄簭鍐欐ц兘寰堥珮锛屽洜涓轰笉闇瑕佷笉鍋滅殑绉诲姩纾佺洏鎸囬拡銆?br /><img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig13.png" height="351" width="698" /><br />HFile鍙傝傿igTable鐨凷STable鍜孒adoop鐨?a >TFile</a>瀹炵幇锛屼粠HBase寮濮嬪埌鐜板湪锛孒File缁忓巻浜嗕笁涓増鏈紝鍏朵腑V2鍦?.92寮曞叆锛孷3鍦?.98寮曞叆銆傞鍏堟垜浠潵鐪嬩竴涓媀1鐨勬牸寮忥細<br /><img src="//www.cqk4s.cn/images/blogjava_net/dlevin/image0080.jpg" alt="" height="160" border="0" width="554" /><br />V1鐨凥File鐢卞涓狣ata Block銆丮eta Block銆丗ileInfo銆丏ata Index銆丮eta Index銆乀railer缁勬垚锛屽叾涓璂ata Block鏄疕Base鐨勬渶灏忓瓨鍌ㄥ崟鍏冿紝鍦ㄥ墠鏂囦腑鎻愬埌鐨凚lockCache灏辨槸鍩轰簬Data Block鐨勭紦瀛樼殑銆備竴涓狣ata Block鐢变竴涓瓟鏁板拰涓绯诲垪鐨凨eyValue(Cell)缁勬垚锛岄瓟鏁版槸涓涓殢鏈虹殑鏁板瓧锛岀敤浜庤〃绀鸿繖鏄竴涓狣ata Block绫诲瀷锛屼互蹇熺洃娴嬭繖涓狣ata Block鐨勬牸寮忥紝闃叉鏁版嵁鐨勭牬鍧忋侱ata Block鐨勫ぇ灏忓彲浠ュ湪鍒涘缓Column Family鏃惰缃?HColumnDescriptor.setBlockSize())锛岄粯璁ゅ兼槸64KB锛屽ぇ鍙风殑Block鏈夊埄浜庨『搴廠can锛屽皬鍙稡lock鍒╀簬闅忔満鏌ヨ锛屽洜鑰岄渶瑕佹潈琛°侻eta鍧楁槸鍙夌殑锛孎ileInfo鏄浐瀹氶暱搴︾殑鍧楋紝瀹冪邯褰曚簡鏂囦欢鐨勪竴浜汳eta淇℃伅锛屼緥濡傦細AVG_KEY_LEN, AVG_VALUE_LEN, LAST_KEY, COMPARATOR, MAX_SEQ_ID_KEY绛夈侱ata Index鍜孧eta Index绾綍浜嗘瘡涓狣ata鍧楀拰Meta鍧楃殑鍏跺疄鐐广佹湭鍘嬬缉鏃跺ぇ灏忋並ey(璧峰RowKey锛?绛夈俆railer绾綍浜咶ileInfo銆丏ata Index銆丮eta Index鍧楃殑璧峰浣嶇疆锛孌ata Index鍜孧eta Index绱㈠紩鐨勬暟閲忕瓑銆傚叾涓璅ileInfo鍜孴railer鏄浐瀹氶暱搴︾殑銆?br /><br />HFile閲岄潰鐨勬瘡涓狵eyValue瀵瑰氨鏄竴涓畝鍗曠殑byte鏁扮粍銆備絾鏄繖涓猙yte鏁扮粍閲岄潰鍖呭惈浜嗗緢澶氶」锛屽苟涓旀湁鍥哄畾鐨勭粨鏋勩傛垜浠潵鐪嬬湅閲岄潰鐨勫叿浣撶粨鏋勶細<br /><img src="//www.cqk4s.cn/images/blogjava_net/dlevin/image0090.jpg" alt="" height="93" border="0" width="553" /><br />寮濮嬫槸涓や釜鍥哄畾闀垮害鐨勬暟鍊硷紝鍒嗗埆琛ㄧずKey鐨勯暱搴﹀拰Value鐨勯暱搴︺傜揣鎺ョ潃鏄疜ey锛屽紑濮嬫槸鍥哄畾闀垮害鐨勬暟鍊硷紝琛ㄧずRowKey鐨勯暱搴︼紝绱ф帴鐫鏄? RowKey锛岀劧鍚庢槸鍥哄畾闀垮害鐨勬暟鍊硷紝琛ㄧずFamily鐨勯暱搴︼紝鐒跺悗鏄疐amily锛屾帴鐫鏄疩ualifier锛岀劧鍚庢槸涓や釜鍥哄畾闀垮害鐨勬暟鍊硷紝琛ㄧずTime Stamp鍜孠ey Type锛圥ut/Delete锛夈俈alue閮ㄥ垎娌℃湁杩欎箞澶嶆潅鐨勭粨鏋勶紝灏辨槸绾补鐨勪簩杩涘埗鏁版嵁浜嗐?strong>闅忕潃HFile鐗堟湰杩佺Щ锛孠eyValue(Cell)鐨勬牸寮忓苟鏈彂鐢熷お澶氬彉鍖栵紝鍙槸鍦╒3鐗堟湰锛屽熬閮ㄦ坊鍔犱簡涓涓彲閫夌殑Tag鏁扮粍</strong>銆?br /> <br />HFileV1鐗堟湰鐨勫湪瀹為檯浣跨敤杩囩▼涓彂鐜板畠鍗犵敤鍐呭瓨澶氾紝骞朵笖Bloom File鍜孊lock Index浼氬彉鐨勫緢澶э紝鑰屽紩璧峰惎鍔ㄦ椂闂村彉闀裤傚叾涓瘡涓狧File鐨凚loom Filter鍙互澧為暱鍒?00MB锛岃繖鍦ㄦ煡璇㈡椂浼氬紩璧锋ц兘闂锛屽洜涓烘瘡娆℃煡璇㈡椂闇瑕佸姞杞藉苟鏌ヨBloom Filter锛?00MB鐨凚loom Filer浼氬紩璧峰緢澶х殑寤惰繜锛涘彟涓涓紝Block Index鍦ㄤ竴涓狧RegionServer鍙兘浼氬闀垮埌鎬诲叡6GB锛孒RegionServer鍦ㄥ惎鍔ㄦ椂闇瑕佸厛鍔犺浇鎵鏈夎繖浜汢lock Index锛屽洜鑰屽鍔犱簡鍚姩鏃堕棿銆備负浜嗚В鍐宠繖浜涢棶棰橈紝鍦?.92鐗堟湰涓紩鍏FileV2鐗堟湰锛?br /><img src="//www.cqk4s.cn/images/blogjava_net/dlevin/hfilev2.png" alt="" height="418" border="0" width="566" /><br />鍦ㄨ繖涓増鏈腑锛孊lock Index鍜孊loom Filter娣诲姞鍒颁簡Data Block涓棿锛岃岃繖绉嶈璁″悓鏃朵篃鍑忓皯浜嗗啓鐨勫唴瀛樹娇鐢ㄩ噺锛涘彟澶栵紝涓轰簡鎻愬崌鍚姩閫熷害锛屽湪杩欎釜鐗堟湰涓繕寮曞叆浜嗗欢杩熻鐨勫姛鑳斤紝鍗冲湪HFile鐪熸琚娇鐢ㄦ椂鎵嶅鍏惰繘琛岃В鏋愩?br /><br />FileV3鐗堟湰鍩烘湰鍜孷2鐗堟湰鐩告瘮锛屽苟娌℃湁澶ぇ鐨勬敼鍙橈紝瀹冨湪KeyValue(Cell)灞傞潰涓婃坊鍔犱簡Tag鏁扮粍鐨勬敮鎸侊紱骞跺湪FileInfo缁撴瀯涓坊鍔犱簡鍜孴ag鐩稿叧鐨勪袱涓瓧娈点傚叧浜庡叿浣揌File鏍煎紡婕斿寲浠嬬粛锛屽彲浠ュ弬鑰?a >杩欓噷</a>銆?br /><br />瀵笻FileV2鏍煎紡鍏蜂綋鍒嗘瀽锛屽畠鏄竴涓灞傜殑绫籅+鏍戠储寮曪紝閲囩敤杩欑璁捐锛屽彲浠ュ疄鐜版煡鎵句笉闇瑕佽鍙栨暣涓枃浠讹細<br /><img alt="" src="//www.cqk4s.cn/images/blogjava_net/dlevin/HBaseArchitecture-Blog-Fig14.png" height="349" width="688" /><br />Data Block涓殑Cell閮芥槸鍗囧簭鎺掑垪锛屾瘡涓猙lock閮芥湁瀹冭嚜宸辩殑Leaf-Index锛屾瘡涓狟lock鐨勬渶鍚庝竴涓狵ey琚斁鍏ntermediate-Index涓紝Root-Index鎸囧悜Intermediate-Index銆傚湪HFile鐨勬湯灏捐繕鏈塀loom Filter鐢ㄤ簬蹇熷畾浣嶉偅涔堟病鏈夊湪鏌愪釜Data Block涓殑Row锛汿imeRange淇℃伅鐢ㄤ簬缁欓偅浜涗娇鐢ㄦ椂闂存煡璇㈢殑鍙傝冦傚湪HFile鎵撳紑鏃讹紝杩欎簺绱㈠紩淇℃伅閮借鍔犺浇骞朵繚瀛樺湪鍐呭瓨涓紝浠ュ鍔犱互鍚庣殑璇诲彇鎬ц兘銆?br /><br />杩欑瘒灏卞厛鍐欏埌杩欓噷锛屾湭瀹屽緟缁傘傘傘?br /><br /> <h2 class="vpdh">鍙傝冿細</h2> https://www.mapr.com/blog/in-depth-look-hbase-architecture#.VdNSN6Yp3qx<br /> //jimbojw.com/wiki/index.php?title=Understanding_Hbase_and_BigTable<br /> //hbase.apache.org/book.html <br /> //www.searchtb.com/2011/01/understanding-hbase.html <br /> //research.google.com/archive/bigtable-osdi06.pdf<img src ="//www.cqk4s.cn/DLevin/aggbug/426877.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="//www.cqk4s.cn/DLevin/" target="_blank">DLevin</a> 2015-08-22 17:44 <a href="//www.cqk4s.cn/DLevin/archive/2015/08/22/426877.html#Feedback" target="_blank" style="text-decoration:none;">鍙戣〃璇勮</a></div>]]></description></item><item><title>Log4J寮曡捣鐨勭▼搴忊滆姝烩?/title><link>//www.cqk4s.cn/DLevin/archive/2015/08/13/426751.html</link><dc:creator>DLevin</dc:creator><author>DLevin</author><pubDate>Thu, 13 Aug 2015 08:28:00 GMT</pubDate><guid>//www.cqk4s.cn/DLevin/archive/2015/08/13/426751.html</guid><wfw:comment>//www.cqk4s.cn/DLevin/comments/426751.html</wfw:comment><comments>//www.cqk4s.cn/DLevin/archive/2015/08/13/426751.html#Feedback</comments><slash:comments>4</slash:comments><wfw:commentRss>//www.cqk4s.cn/DLevin/comments/commentRss/426751.html</wfw:commentRss><trackback:ping>//www.cqk4s.cn/DLevin/services/trackbacks/426751.html</trackback:ping><description><![CDATA[<h2 class="vpdh">闂璧峰洜</h2> 渚濈劧鏄湪浣跨敤GemFire鐨勯泦缇や腑锛屾垜浠彂鐜板伓灏斾細鍑虹幇涓浜汫emFire鐨凢unction鎵ц鐗瑰埆鎱紝骞朵笖瓒呰繃浜嗕袱鍒嗛挓锛堜负浜嗕繚璇佹暟鎹殑涓鑷存э紝鎴戜滑鍦ㄥ啓涔嬪墠闇瑕佸厛鎷夸竴涓狶ock锛屽洜涓轰笉鑳芥瘡涓狵ey閮藉搴斾竴涓狶ock锛屽洜鑰屾垜浠娇鐢ㄤ簡Guava鐨凷tripe Lock锛堝叧浜嶴tripe Lock鍙互鍙傝?a href="//www.cqk4s.cn/DLevin/archive/2013/12/25/407990.html">杩欓噷</a>锛夛紝鑰屼笖杩欎釜Lock鏈韩鎴戜滑鎸囧畾浜?鍒嗛挓鐨勮秴鏃舵椂闂达紝鍥犺屽鏋滃啓瓒呰繃涓ゅ垎閽燂紝鎴戜滑灏变細鏀跺埌Exception锛夈傝繖涓棶棰樺叾瀹炲凡缁忓洶鎵颁簡鎴戜滑濂藉嚑骞翠簡锛屽垰鍓嶆鏃堕棿锛屾垜浠彂鐜伴暱鏃堕棿鐨凷top-The-World GC浼氬紩璧疯繖涓棶棰橈紝鑰屼笖杩欑鏃跺欏緢澶氭椂鍊欎細寮曡捣閭d釜鑺傜偣浠庨泦缇や腑閫鍑猴紝骞朵笉鏄墍鏈夌殑杩欑閿欒閮芥湁GC鐨勯棶棰橈紝鎴戠壒鍦版煡浜咷C鐨勬棩蹇楋紝鏈変簺杩欑鍐欒秴杩囦袱鍒嗛挓鐨勬儏鍐典笅锛孏C涓鐩村浜庨潪甯稿仴搴风殑鐘舵侊紝鑰屼笖鏌ヤ簡GemFire鐨勬棩蹇楀拰鎴戜滑鑷繁鐨勬棩蹇楋紝涔熸病鏈夊彂鐜颁换浣曞紓甯搞傜敱浜庢垜浠瘡涓暟鎹繚鐣欎袱鍒嗕唤鎷疯礉锛屼篃灏辨槸璇存瘡娆℃暟鎹啓閮借鍐欎袱涓妭鐐癸紝涓ゅ垎閽熷CPU鏉ヨ鍙互鍋氬お澶氱殑浜嬫儏锛屽洜鑰屽彧鏈塈O鎵嶈兘鍦ㄦ煇浜涙椂鍊欎骇鐢熻繖绉嶉棶棰橈紝鍦ㄩ棶棰樺彂鐢熺殑鏃跺欎篃娌℃湁浠讳綍overflow鏁版嵁锛岃屼笖鏈湴鎿嶄綔锛屽嵆浣垮IO鏉ヨ2鍒嗛挓涔熸槸涓涓潪甯搁暱鐨勬椂闂翠簡锛屽洜鑰屾垜浠彧鑳芥鐤戣繖鏄啓鍙︿竴涓妭鐐瑰紩璧风殑锛屽鍙︿竴涓妭鐐癸紝瀹冩槸鍦ㄥ悓涓涓狣ata Center涓紝鑰屼笖鍩烘湰鏄湪鍚屼竴涓狢hasis鍐呴儴锛屽洜鑰屽畠浠箣闂村皬浜?M鐨勬暟鎹噺閫氫俊涔熶笉澶彲鑳借姳鍘?鍒嗛挓鐨勬椂闂达紝鎵浠ュ墿涓嬬殑鎴戜滑灏卞彧鑳芥鐤戠綉缁滅殑闂浜嗭紝姣斿鏁版嵁涓㈠寘銆佺綉缁滄姈鍔ㄣ佺綉缁滄祦閲忓お澶т竴璧蜂紶杈撳彉鎱㈢瓑锛屼絾鏄垜浠病鏈夋壘鍒颁换浣曠浉鍏崇殑闂銆傛墍浠ユ垜浠緢闀夸竴娈垫椂闂寸礌鎵嬫棤绛栵紝鍙兘鎬狦emFire闂簮锛屾垜浠笉鐭ラ亾杩欎袱鍒嗛挓鏄笉鏄疓emFire鑷繁鍐呴儴鍦ㄥ仛涓浜涗笉涓轰汉鐭ョ殑浜嬫儏锛屽洜鑰屽お蹇欎簡鑰屾瘡鏉ュ緱鍙婂鐞嗘垜浠殑鍐欒姹傘傝櫧鐒舵垜涓鐩磋寰椾笉绠″湪澶勭悊浠涔堢倰浣滐紝涓ゅ垎閽熼兘娌℃湁鍝嶅簲鏍规湰鏃犳硶瑙i噴鐨勯氾紝鏇翠綍鍐礕emFire鑺傜偣涔嬮棿骞舵病鏈夋姤鍛婃湁浠讳綍寮傚父锛屾垨鑰呭儚浠ュ墠鍙戠幇鐨勪竴涓妭鐐瑰悜Locator涓炬姤鍙︿竴涓妭鐐规病鏈夊搷搴旂殑闂锛孡ocator鑷繁涔熻兘寰堟甯哥殑鍚戦偅涓妭鐐瑰彂閫佹柊鐨勬垚鍛樹俊鎭紙View锛夛紝鍥犺岀湅璧锋潵鍚戞槸杩欎釜鑺傜偣铏界劧鑺变簡涓ゅ垎閽熷鏉ュ啓涓涓暟鎹紝浣嗘槸瀹冭繕鏄湁鍝嶅簲鐨勶紝鏈夌偣“鍋囨”鐨勮刀鑴氥?br /> <br /> <h2 class="vpdh">闂鍙戠幇</h2> 杩欎釜闂杩欎箞鍑犲勾浠ユ潵鏃朵笉鏃剁殑灏变細鍙戠敓锛岃屼笖鍥犱负浠ュ墠鑺辩殑鏃堕棿澶浜嗭紝鑰屼笖涔熸病鏈夋壘鍒颁换浣曞嚭閿欑殑鍦版柟锛岀幇鍦ㄧ储鎬т笉鍘昏姳澶鏃堕棿鍦ㄤ笂闈簡锛屾洿浣曞喌杩欎釜瀹冨緢闀挎椂闂存墠鍙戠敓涓娆★紝骞朵笖浠婂勾浠ユ潵灏变竴鐩存病鍙戠敓杩囷紝鐩村埌鍓嶅嚑鍛ㄥ嚭鐜颁竴娆★紝鎴戞湁鐐逛笉淇¢偑鐨勯噸鏂板幓鐪嬭繖涓棶棰橈紝渚濈劧娌℃湁鎵惧埌浠讳綍鍙枒鐨勫湴鏂癸紝GC鏃ュ織銆佸簲鐢ㄧ▼搴忔棩蹇椼丟emFire鑷繁鐨勬棩蹇椼佺綉缁溿丆PU浣跨敤鎯呭喌绛夋墍鏈夌殑閮芥槸姝e父鐨勶紝闄や簡闂鍙戠敓鐨勯偅涓椂鍒伙紝搴旂敤绋嬪簭娌℃湁浠讳綍鏃ュ織锛屽彟澶栧湪闂鍙戠敓涔嬪墠鍑虹幇杩嘗og4J鏃ュ織鏂囦欢鐨凴olling锛堟垜浠娇鐢≧ollingFileAppender锛屽苟涓斿彧淇濈暀20涓棩蹇楁枃浠讹級锛屼絾鏄疞og4J鏃ュ織鏂囦欢Roll鐨勬棩蹇楀嚭鐜颁簡鏂粨锛屽湪寮濮嬭Roll鍒扮湡姝e畬鎴怰oll涓棿杩樻湁鍑犺GemFire鑷韩鐨勬棩蹇楋紝姝ゆ椂鎴戝苟娌℃湁瑙夊緱杩欎釜鏄湁寰堝ぇ闂鐨勶紝鍥犱负鎴戝缁堣寰桳og4J闄や簡瀹冭嚜宸辨彁鍒板钩鍧囧鎬ц兘鏈?0%鐨勫奖鍝嶄互澶栵紝瀹冨氨鏄竴涓畝鍗曠殑鎶婃棩蹇楀啓鍒版枃浠剁殑杩囩▼锛屼笉浼氬奖鍝嶇殑鏁翠釜搴旂敤绋嬪簭鏈韩锛屽洜涓哄畠澶畝鍗曚簡锛岀洿鍒颁粖澶╄繖涓棶棰樺啀娆″嚭鐜帮紝渚濈劧娌℃湁浠讳綍鍏朵粬鏂归潰鐨勬敹鑾凤紝鎵鏈夌殑鍦版柟閮芥樉绀烘甯哥姸鎬侊紝鐢氳嚦鎴戜滑涔嬪墠鍙戠幇鐨勭綉鍗¢棶棰樹粖澶╀篃娌℃湁鍙戠敓锛岀劧鑰屽悓鏍锋槸鍑洪棶棰樼殑涓ゅ垎閽熸病鏈夊嚭鐜板簲鐢ㄧ▼搴忔棩蹇楋紝鏃ュ織鏂囦欢Roll鐨勬棩蹇楀拰涓婃绫讳技锛屽紑濮婻oll鍒扮粨鏉熷嚭鐜癎emFire鏃ュ織鐨勪氦鍙夈? <div class="vpdh"><fieldset><legend>鏈杩戜竴娆″彂鐢熺殑鏃ュ織</legend> <div class="vpdh">[info 2015/08/12 01:56:07.736 BST …] ClientHealthMonitor: Registering client with member id …</div> <div class="vpdh">log4j: rolling over count=20971801</div> <div class="vpdh">log4j: maxBackupIndex=20</div> <div class="vpdh">[info 2015/08/12 01:56:12.265 BST …] ClientHealthMonitor: Unregistering client with member id …</div> <div class="vpdh">……</div> <div class="vpdh">[info 2015/08/12 01:56:23.773 BST …] ClientHealthMonitor: Registering client with member id …</div> <div class="vpdh">log4j: Renaming file logs/….log.19 to logs/….log.20</div> </fieldset></div> <div class="vpdh"><fieldset><legend>涓鍛ㄥ墠鍙戠敓鐨勬棩蹇?/legend> <div class="vpdh">[info 2015/08/04 01:43:45.761 BST …] ClientHealthMonitor: Registering client with member id …</div> <div class="vpdh">log4j: rolling over count=20971665</div> <div class="vpdh">log4j: maxBackupIndex=20</div> <div class="vpdh">……</div> <div class="vpdh">[info 2015/08/04 01:45:25.506 BST …] ClientHealthMonitor: Registering client with member id …</div> <div class="vpdh">log4j: Renaming file logs/….log.19 to logs/….log.20</div> </fieldset></div> <div class="vpdh">鐪嬩技杩欎釜鏄竴涓寰嬶紙濂楃敤鍚屼簨鐨勪竴鍙ヨ瘽锛氫竴娆″彂鐢熸椂鍋剁劧锛屼袱娆″彂鐢熷氨鏄瀛︿簡锛夈傜劧鑰屾鏃舵垜鍏跺疄渚濈劧涓嶅お鐩镐俊Log4J鏄?#8220;鍑舵墜”锛屽洜涓烘垜涓鐩磋寰桳og4J鏄竴涓畝鍗曠殑鏃ュ織杈撳嚭妗嗘灦锛屽畠瑕佹槸鍑洪棶棰樹篃鍙槸瀹冭嚜宸辩殑闂锛屾槸灞閮ㄧ殑锛岃岃繖涓棶棰樼殑鍑虹幇鏄庢樉鏄叏灞鐨勶紝鐩村埌鎴戠獊鐒惰剳瀛愪竴闂岃繃锛?strong>鏃ュ織鎵撳嵃鐨勬搷浣滄槸synchronized锛屼篃灏辨槸璇村湪鏃ュ織鏂囦欢Roll鐨勬椂鍊欙紝鎵鏈夊叾瀹冮渶瑕佹墦鏃ュ織鐨勭嚎绋嬮兘瑕佺瓑寰呯洿鍒癛oll瀹屾垚锛屽鏋滆繖涓猂oll杩囩▼瓒呰繃浜?鍒嗛挓锛岄偅涔堝氨浼氬彂鐢熸垜浠湅鍒扮殑Stripe Lock瓒呮椂锛屼篃灏辨槸鍙戠敓浜嗙▼搴?#8220;鍋囨”鐨勭姸鎬併?/strong>閲嶆柊鏌ョ湅Log4J鎵撳嵃鏃ュ織鐨勬柟娉曡皟鐢ㄦ爤锛屽畠浼氬湪涓や釜鍦版柟鐢╯ynchronized锛屽嵆鍚屼竴涓狢ategory锛圠ogger锛夌被瀹炰緥锛?br /> <div style="font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%; word-break: break-all; background-color: #eeeeee;"><!--<br /> <br /> Code highlighting produced by Actipro CodeHighlighter (freeware)<br /> //www.CodeHighlighter.com/<br /> <br /> -->    <span style="color: #0000FF; ">public</span> <span style="color: #0000FF; ">void</span> callAppenders(LoggingEvent event) {<br />         <span style="color: #0000FF; ">int</span> writes = 0;<br />         <span style="color: #0000FF; ">for</span>(Category c = <span style="color: #0000FF; ">this</span>; c != <span style="color: #0000FF; ">null</span>; c=c.parent) {<br />             <span style="color: #008000; ">//</span><span style="color: #008000;"> Protected against simultaneous call to addAppender, removeAppender,<img src="//www.cqk4s.cn/Images/dot.gif" alt="" /></span><span style="color: #008000; "><br /> </span>            <span style="color: #0000FF; ">synchronized</span>(c) {<br />                 <span style="color: #0000FF; ">if</span>(c.aai != <span style="color: #0000FF; ">null</span>) {<br />                     writes += c.aai.appendLoopOnAppenders(event);<br />                 }<br />                 <span style="color: #0000FF; ">if</span>(!c.additive) {<br />                     <span style="color: #0000FF; ">break</span>;<br />                 }<br />             }<br />         }<br /> 銆傘傘?br />     }</div> </div>浠ュ強鍚屼竴涓狝ppender鍦╠oApppend鏃讹細<br /><div style="font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%; word-break: break-all; background-color: #eeeeee;"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />//www.CodeHighlighter.com/<br /><br />-->    <span style="color: #0000FF; ">public</span> <span style="color: #0000FF; ">synchronized</span> <span style="color: #0000FF; ">void</span> doAppend(LoggingEvent event) {<br />      銆傘傘?br />      <span style="color: #0000FF; ">this</span>.append(event);<br />    }</div><div class="vpdh">鑰孯oll鐨勮繃绋嬪氨鏄湪append鏂规硶涓紝杩涗竴姝ュ垎鏋愶紝鍦ㄤ笅闈袱鍙ヨ瘽涔嬮棿锛屼粬浠垎鍒姳璐逛簡瓒呰繃100s鍜岃秴杩?1s鐨勬椂闂达細</div><div class="vpdh">log4j: maxBackupIndex=20<br />銆傘傘?/div><div class="vpdh">log4j: Renaming file logs/….log.19 to logs/….log.20</div><div class="vpdh">鑰岃繖涓ゅ彞涔嬮棿鍙寘鍚簡涓や釜File.exists()锛屼竴涓狥ile.delete()锛屼竴涓狥ile.rename()鎿嶄綔锛?/div><div style="font-size: 13px; border: 1px solid #cccccc; padding: 4px 5px 4px 4px; width: 98%; word-break: break-all; background-color: #eeeeee;"><!--<br /><br />Code highlighting produced by Actipro CodeHighlighter (freeware)<br />//www.CodeHighlighter.com/<br /><br />-->    <span style="color: #0000FF; ">public</span> <span style="color: #0000FF; ">void</span> rollOver() {<br />      銆傘傘?br />      <span style="color: #0000FF; ">if</span>(maxBackupIndex > 0) {<br />        <span style="color: #008000; ">//</span><span style="color: #008000;"> Delete the oldest file, to keep Windows happy.</span><span style="color: #008000; "><br /></span>        file = <span style="color: #0000FF; ">new</span> File(fileName + '.' + maxBackupIndex);<br />        <span style="color: #0000FF; ">if</span> (file.exists())<br />            renameSucceeded = file.delete();<br />        <span style="color: #0000FF; ">for</span> (<span style="color: #0000FF; ">int</span> i = maxBackupIndex - 1; i >= 1 && renameSucceeded; i--) {<br />            file = <span style="color: #0000FF; ">new</span> File(fileName + "." + i);<br />            <span style="color: #0000FF; ">if</span> (file.exists()) {<br />                target = <span style="color: #0000FF; ">new</span> File(fileName + '.' + (i + 1));<br />                LogLog.debug("Renaming file " + file + " to " + target);<br />                renameSucceeded = file.renameTo(target);<br />            }<br />        }<br />      銆傘傘?br />      }<br />    }</div><div class="vpdh"><h2 class="vpdh">NFS绠鍗曟ц兘娴嬭瘯鍜屽垎鏋?/h2>鍥犺屾垜瀵筃FS鐨勬ц兘浣滀簡涓浜涚畝鍗曟祴璇曪細</div><div class="vpdh">鍙湁涓涓嚎绋嬫椂锛屽湪NFS涓媟ename鎬ц兘锛?/div><div class="vpdh">1 file:                    3ms</div><div class="vpdh">10 files:                48ms</div><div class="vpdh">20 files:                114ms</div><div class="vpdh">鐩告瘮杈冿紝鍦ㄦ湰鍦扮鐩榬ename鐨勬ц兘锛?/div><div class="vpdh">1 file:                    1ms</div><div class="vpdh">3 files:                  1ms</div><div class="vpdh">10 files:                3ms</div><div class="vpdh">瀵筃FS鍜屾湰鍦扮鐩樺啓鐨勬ц兘锛堟ā鎷熸棩蹇楋紝姣忚閮戒細flush锛夛細</div><table border="0" cellspacing="0" cellpadding="0" style="border-collapse:collapse;"> <tbody><tr style="height:13.4pt"> <td width="139" valign="top" style="width: 1.45in; border: 1pt solid windowtext; padding: 0in 5.4pt; height: 13.4pt;"> <p> </p> </td> <td width="78" valign="top" style="width: 58.5pt; border-style: solid solid solid none; border-top-color: windowtext; border-right-color: windowtext; border-bottom-color: windowtext; border-top-width: 1pt; border-right-width: 1pt; border-bottom-width: 1pt; padding: 0in 5.4pt; height: 13.4pt;"> <p><span style="color:#1F497D">NFS</span></p> </td> <td width="78" valign="top" style="width: 58.5pt; border-style: solid solid solid none; border-top-color: windowtext; border-right-color: windowtext; border-bottom-color: windowtext; border-top-width: 1pt; border-right-width: 1pt; border-bottom-width: 1pt; padding: 0in 5.4pt; height: 13.4pt;"> <p><span style="color:#1F497D">LOCAL</span></p> </td> </tr> <tr> <td width="139" valign="top" style="width: 1.45in; border-style: none solid solid; border-right-color: windowtext; border-bottom-color: windowtext; border-left-color: windowtext; border-right-width: 1pt; border-bottom-width: 1pt; border-left-width: 1pt; padding: 0in 5.4pt;"> <p><span style="color:#1F497D">1 writer, 11M</span></p> </td> <td width="78" valign="top" style="width:58.5pt;border-top:none;border-left:none; border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt; padding:0in 5.4pt 0in 5.4pt"> <p><span style="color:#1F497D">443ms</span></p> </td> <td width="78" valign="top" style="width:58.5pt;border-top:none;border-left:none; border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt; padding:0in 5.4pt 0in 5.4pt"> <p><span style="color:#1F497D">238ms</span></p> </td> </tr> <tr> <td width="139" valign="top" style="width: 1.45in; border-style: none solid solid; border-right-color: windowtext; border-bottom-color: windowtext; border-left-color: windowtext; border-right-width: 1pt; border-bottom-width: 1pt; border-left-width: 1pt; padding: 0in 5.4pt;"> <p><span style="color:#1F497D">1 writer, 101M</span></p> </td> <td width="78" valign="top" style="width:58.5pt;border-top:none;border-left:none; border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt; padding:0in 5.4pt 0in 5.4pt"> <p><span style="color:#1F497D">2793ms</span></p> </td> <td width="78" valign="top" style="width:58.5pt;border-top:none;border-left:none; border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt; padding:0in 5.4pt 0in 5.4pt"> <p><span style="color:#1F497D">992ms</span></p> </td> </tr> <tr> <td width="139" valign="top" style="width: 1.45in; border-style: none solid solid; border-right-color: windowtext; border-bottom-color: windowtext; border-left-color: windowtext; border-right-width: 1pt; border-bottom-width: 1pt; border-left-width: 1pt; padding: 0in 5.4pt;"> <p><span style="color:#1F497D">10 writers, 11M</span></p> </td> <td width="78" valign="top" style="width:58.5pt;border-top:none;border-left:none; border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt; padding:0in 5.4pt 0in 5.4pt"> <p><span style="color:#1F497D">~4400ms</span></p> </td> <td width="78" valign="top" style="width:58.5pt;border-top:none;border-left:none; border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt; padding:0in 5.4pt 0in 5.4pt"> <p><span style="color:#1F497D">~950ms</span></p> </td> </tr> <tr> <td width="139" valign="top" style="width: 1.45in; border-style: none solid solid; border-right-color: windowtext; border-bottom-color: windowtext; border-left-color: windowtext; border-right-width: 1pt; border-bottom-width: 1pt; border-left-width: 1pt; padding: 0in 5.4pt;"> <p><span style="color:#1F497D">10 writers, 101M</span></p> </td> <td width="78" valign="top" style="width:58.5pt;border-top:none;border-left:none; border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt; padding:0in 5.4pt 0in 5.4pt"> <p><span style="color:#1F497D">~30157ms</span></p> </td> <td width="78" valign="top" style="width:58.5pt;border-top:none;border-left:none; border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt; padding:0in 5.4pt 0in 5.4pt"> <p><span style="color:#1F497D">~5500ms</span></p> </td> </tr></tbody></table><div class="vpdh"><br />涓浜涘叾浠栫殑缁熻锛?/div><div class="vpdh"><strong class="vpdh">100鍚屾椂鍐?</strong></div><div class="vpdh">Create 20 files spend: 301ms</div><div class="vpdh">Renaming 20 files spends: 333ms</div><div class="vpdh">Delete 20 files spends: 329ms</div><div class="vpdh"></div><div class="vpdh"><strong class="vpdh">1000鍚屾椂鍐?</strong></div><div class="vpdh">Create 20 files spend: 40145ms</div><div class="vpdh">Renaming 20 files spends: 39273ms<br /></div><div class="vpdh"><strong class="vpdh">鑰屽湪1000涓悓鏃跺啓鐨勮繃绋嬩腑锛岄噸鍛藉悕锛?/strong></div><div class="vpdh">Rename file: LogTest1.50 take: <strong class="vpdh">36434ms</strong></div><div class="vpdh">Rename file: LogTest1.51 take: <strong class="vpdh">39ms</strong></div><div class="vpdh">Rename file: LogTest1.52 take: <strong class="vpdh">34ms</strong><br /></div><div class="vpdh">涔熷氨鏄鍦ㄨ繖涓ā鎷熻繃绋嬩腑锛屼竴涓枃浠剁殑rename瓒呰繃36s锛岃屽悜鎴戜滑鏈夊崄鍑犲彴鏈哄櫒鍚屾椂浣跨敤鐩稿悓鐨凬FS锛屽苟涓旀瘡鍙版満鍣ㄤ笂閮借窇浜屼笁鍗佷釜绋嬪簭锛屽鏋滈偅娈垫椂闂村悓鏃舵湁涓婁竾涓殑鏃ュ織鍐欙紝鍙互棰勮杈惧埌100s鎯呭喌鏄彲鑳藉彂鐢熺殑銆?/div><div class="vpdh">鍏充簬NFS鎬ц兘鐨勯棶棰橈紝鍦ㄣ婃瀯寤洪珮鎬ц兘WEB绔欑偣銆嬬殑涔︼紙330椤碉級涓篃鏈夋秹鍙娿傜畝鍗曠殑浠嬬粛锛孨FS鐢盨un鍦?984骞村紑鍙戯紝鏄富娴佸紓鏋勫钩鍙板疄鐜版枃浠跺叡浜殑棣栭夋柟妗堛傚畠骞舵病鏈夎嚜宸辩殑浼犺緭鍗忚锛岃屾槸浣跨敤RPC锛圧emote Procedure Call锛夊崗璁紙搴旂敤灞傦級锛孯PC鍗忚榛樿搴曞眰鍩轰簬UDP浼犺緭锛屼絾鏄嚜宸卞疄鐜板湪涓㈠寘鏃剁殑閲嶄紶鏈哄埗锛岃屼笖NFS鏈嶅姟鍣ㄩ噰鐢ㄥ杩涚▼妯″瀷锛岄粯璁よ繘绋嬩负4锛屼絾鏄竴鑸兘浼氳皟浼樺鍔犳湇鍔¤繘绋嬫暟锛岀劧鑰?#8220;涓嶇鎬庝箞瀵筃FS杩涜鎬ц兘浼樺寲锛孨FS娉ㄥ畾涓嶉傚悎浣滀负I/O瀵嗛泦鍨嬫枃浠跺叡浜柟妗堬紝浣嗗彲浠ヤ綔涓轰竴鑸敤閫旓紝姣斿鎻愪緵绔欑偣鍐呴儴鐨勮祫婧愬叡浜紝瀹冪殑浼樺娍鍦ㄤ簬瀹规槗鎼缓锛岃屼笖鍙互鍑忓皯涓嶅繀瑕佺殑鏁版嵁鍐椾綑銆?#8221;</div><div class="vpdh">鍙互浣跨敤鍛戒护锛?#8220;nfsstat -c”鑾峰彇瀵筃FS鏈嶅姟鍣ㄧ殑鎿嶄綔鐨勭畝鍗曠粺璁★紝鍏蜂綋鍙互鍙傝冦婃瀯寤洪珮鎬ц兘WEB绔欑偣銆嬬殑鐩稿叧绔犺妭锛岄噷闈㈣繕鏈夋洿璇︾粏鐨勫NFS鏈嶅姟鍣ㄦц兘鐨勬祴璇曘?/div><div class="vpdh"><br /><h2 class="vpdh">鎬荤粨</h2><strong class="vpdh">浠庤繖涓簨浠舵垜鎬荤粨浜嗕袱浠朵簨鎯咃細</strong></div><div class="vpdh">1. 鏃ュ織鐨勫奖鍝嶅彲鑳芥槸鍏ㄥ眬鎬х殑锛屽洜鑰岃闈炲父灏忓績锛屼竴涓楁椂鐨勬搷浣滃彲鑳藉紩璧风▼搴忕殑“鍋囨”锛屽洜鑰岃闈炲父灏忓績銆?/div><div class="vpdh">2. 铏界劧鎶婃棩蹇楁墦鍗板湪NFS涓婏紝瀵瑰ぇ閲忕殑鏃ュ織鏂囦欢鏌ユ壘浼氭柟渚垮緢澶氾紝浣嗘槸杩欐槸涓涓緢鑰楁ц兘鐨勮璁★紝鐗瑰埆鏄綋澶ч噺鐨勭▼搴忓叡浜繖涓狽FS鐨勬椂鍊欙紝鍥犺岃灏介噺閬垮厤銆?/div><img src ="//www.cqk4s.cn/DLevin/aggbug/426751.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="//www.cqk4s.cn/DLevin/" target="_blank">DLevin</a> 2015-08-13 16:28 <a href="//www.cqk4s.cn/DLevin/archive/2015/08/13/426751.html#Feedback" target="_blank" style="text-decoration:none;">鍙戣〃璇勮</a></div>]]></description></item><item><title>瀹炵幇鑷繁鐨凩ock瀵硅薄 - ▼幸运赛车规则▼幸运赛车投注▼幸运赛车开奖结果查询▼湖南幸运赛车开奖奖金▼幸运赛车走势▼//www.cqk4s.cn/DLevin/archive/2015/08/11/426723.htmlDLevinDLevinMon, 10 Aug 2015 22:08:00 GMT//www.cqk4s.cn/DLevin/archive/2015/08/11/426723.html//www.cqk4s.cn/DLevin/comments/426723.html//www.cqk4s.cn/DLevin/archive/2015/08/11/426723.html#Feedback0//www.cqk4s.cn/DLevin/comments/commentRss/426723.html//www.cqk4s.cn/DLevin/services/trackbacks/426723.html涓鐩存兂濂藉ソ瀛︿範concurrent鍖呬腑鐨勫悇涓被鐨勫疄鐜帮紝鐒惰岀粡甯哥湅浜嗕竴鐐瑰氨鍥犱负鍏朵粬浜嬫儏骞叉壈鑰屾斁涓嬩簡銆傚彂鐜拌繖鏍峰お涓嶅埄浜庤嚜宸辩殑鎴愰暱浜嗭紝鍥犺屾渶杩戞墦绠楁綔蹇冧竴浠朵竴浠剁殑瀹屾垚鑷繁鎯冲涔犵殑涓滆タ銆?br />
瀵筩oncurrent鍖呯殑瀛︿範鎵撶畻鍏堜粠Lock鐨勫疄鐜板紑濮嬶紝鍥犺岃嚜鐒惰岀劧鐨勫氨绔捣浜咥bstractQueuedSynchronizer锛岀劧鑰岃璇绘噦杩欎釜绫荤殑婧愮爜骞朵笉鏄偅涔堝鏄擄紝鍥犺屾垜灏卞紑濮嬮棶鑷繁涓涓棶棰橈細濡傛灉鑷繁瑕佸幓瀹炵幇杩欎釜涓涓狶ock瀵硅薄锛屽簲璇ュ浣曞疄鐜板憿锛?br />
瑕佸疄鐜癓ock瀵硅薄锛岄鍏堢悊瑙d粈涔堟槸閿侊紵鎴戣嚜宸变粠缂栫▼瑙掑害绠鍗曠殑鐞嗚В锛屾墍璋撻攣瀵硅薄锛堜簰鏂ラ攣锛夊氨鏄畠鑳戒繚璇佷竴娆″彧鏈変竴涓嚎绋嬭兘杩涘叆瀹冧繚鎶ょ殑涓寸晫鍖猴紝濡傛灉鏈変竴涓嚎绋嬪凡缁忔嬁鍒伴攣瀵硅薄锛岄偅涔堝叾浠栧璞″繀椤昏鏉冪瓑寰咃紝鑰屽湪璇ョ嚎绋嬮鍑鸿繖涓复鐣屽尯鏃堕渶瑕佸敜閱掔瓑寰呭垪琛ㄤ腑鐨勫叾浠栫嚎绋嬨傛洿瀛︽湳涓浜涳紝銆婅绠楁満鎿嶄綔绯荤粺銆?/a>涓鍚屾鏈哄埗鍑嗗垯鐨勫綊绾筹紙P50锛夛細

  1. 绌洪棽璁╄繘銆傚綋鏃犺繘绋嬪浜庝复鐣屽尯鏃讹紝琛ㄦ槑涓寸晫璧勬簮澶勪簬绌洪棽鐘舵侊紝搴斿厑璁镐竴涓姹傝繘鍏ヤ复鐣屽尯鐨勮繘绋嬬珛鍗宠繘鍏ヨ嚜宸辩殑涓寸晫鍖猴紝浠ユ湁鏁堢殑鍒╃敤涓寸晫璧勬簮銆?/li>
  2. 蹇欏垯绛夊緟銆傚綋宸叉湁杩涚▼杩涘叆涓寸晫鍖烘椂锛岃〃鏄庝复鐣岃祫婧愭鍦ㄨ璁块棶锛屽洜鑰屽叾浠栬瘯鍥捐繘鍏ヤ复鐣屽尯鐨勮繘绋嬪繀椤荤瓑寰咃紝浠ヤ繚璇佸涓寸晫鍖鸿祫婧愮殑浜掓枼璁块棶銆?/li>
  3. 鏈夐檺绛夊緟銆傚瑕佹眰璁块棶涓寸晫璧勬簮鐨勮繘绋嬶紝搴斾繚璇佸湪鏈夐檺鏃堕棿鍐呰兘杩涘叆鑷繁鐨勪复鐣屽尯锛屼互鍏嶉櫡鍏?#8220;姝荤瓑”鐘舵併?/li>
  4. 璁╂潈绛夊緟銆傚綋杩涚▼涓嶈兘杩涘叆鑷繁鐨勪复鐣屽尯鏃讹紝搴旇閲婃斁澶勭悊鏈猴紝浠ュ厤杩涚▼闄峰叆“蹇欑瓑”鐘舵併?/li>

璇翠簡閭d箞澶氾紝鍏跺疄瀵逛簰鏂ラ攣寰堢畝鍗曪紝鍙渶瑕佷竴涓爣璁颁綅锛屽鏋滆鏍囪浣嶄负0锛岃〃绀烘病鏈夎鍗犵敤锛屽洜鑰岀洿鎺ヨ幏寰楅攣锛岀劧鍚庢妸璇ユ爣璁颁綅缃负1锛屾鏃跺叾浠栫嚎绋嬪彂鐜拌鏍囪浣嶅凡缁忔槸1锛屽洜鑰岄渶瑕佺瓑寰呫傝繖閲屽杩欎釜鏍囪浣嶇殑姣旇緝骞惰鍊煎繀椤绘槸鍘熷瓙鎿嶄綔锛岃屽湪JDK5浠ュ悗鎻愪緵鐨刟tomic鍖呴噷鐨勫伐鍏风被鍙互寰堟柟渚跨殑鎻愪緵杩欎釜鍘熷瓙鎿嶄綔銆傜劧鑰屼笂闈㈢殑鍥涗釜鍑嗗垯搴旇婕忎簡涓鐐癸紝鍗抽噴鏀鹃攣鐨勭嚎绋嬶紙杩涚▼锛夊拰寰楀埌閿佺殑绾跨▼锛堣繘绋嬶級搴旇鏄悓涓涓紝灏卞儚涓鎶婇挜鍖欏搴斾竴鎶婇攣锛堢悊鎯崇殑锛夛紝鎵浠ヤ竴涓潪甯哥畝鍗曠殑Lock绫诲彲浠ヨ繖涔堝疄鐜帮細

public class SpinLockV1 {
    
private final AtomicInteger state = new AtomicInteger(0);
    
private volatile Thread owner; // 杩欓噷owner瀛楁鍙兘瀛樺湪涓棿鍊硷紝涓嶅彲闈狅紝鍥犺屽叾浠栫嚎绋嬩笉鍙互渚濊禆杩欎釜瀛楁鐨勫?/span>
    
    
public void lock() {
        
while (!state.compareAndSet(01)) { }
        owner 
= Thread.currentThread();
    }
    
    
public void unlock() {
        Thread currentThread 
= Thread.currentThread();
        
if (owner != currentThread || !state.compareAndSet(10)) {
            
throw new IllegalStateException("The lock is not owned by thread: " + currentThread);
        }
        owner 
= null;
    }
}

涓涓畝鍗曠殑娴嬭瘯鏂规硶锛?br />

    @Test
    
public void testLockCorrectly() throws InterruptedException {
        
final int COUNT = 100;
        Thread[] threads 
= new Thread[COUNT];
        SpinLockV1 lock 
= new SpinLockV1();
        AddRunner runner 
= new AddRunner(lock);
        
for (int i = 0; i < COUNT; i++) { 
            threads[i] 
= new Thread(runner, "thread-" + i);
            threads[i].start();
        }
        
        
for (int i = 0; i < COUNT; i++) {
            threads[i].join();
        }
        
        assertEquals(COUNT, runner.getState());
    }
    
    
private static class AddRunner implements Runnable {
        
private final SpinLockV1 lock;
        
private int state = 0;

        
public AddRunner(SpinLockV1 lock) {
            
this.lock = lock;
        }
        
        
public void run() {
            lock.lock();
            
try {
                quietSleep(
10);
                state
++;
                System.out.println(Thread.currentThread().getName() 
+ "" + state);
            } 
finally {
                lock.unlock();
            }
        }
        
        
public int getState() {
            
return state;
        }
    }

鐒惰岃繖涓猄pinLock鍏跺疄骞朵笉闇瑕乻tate杩欎釜瀛楁锛屽洜涓簅wner鐨勮祴鍊间笌鍚︿篃鏄竴绉嶇姸鎬侊紝鍥犺屽彲浠ョ敤瀹冧綔涓轰竴绉嶄簰鏂ョ姸鎬侊細

public class SpinLockV2 {
    
private final AtomicReference<Thread> owner = new AtomicReference<Thread>(null);
    
    
public void lock() {
        
final Thread currentThread = Thread.currentThread();
        
while (!owner.compareAndSet(null, currentThread)) { }
    }
    
    
public void unlock() {
        Thread currentThread 
= Thread.currentThread();
        
if (!owner.compareAndSet(currentThread, null)) {
            
throw new IllegalStateException("The lock is not owned by thread: " + currentThread);
        }
    }
}

杩欏湪鎿嶄綔绯荤粺涓瀹氫箟涓烘暣褰俊鍙烽噺锛岀劧鑰屾暣褰俊鍙烽噺濡傛灉娌℃嬁鍒伴攣浼氫竴鐩村浜?#8220;蹇欑瓑”鐘舵侊紙娌℃湁閬靛惊鏈夐檺绛夊緟鍜岃鏉冪瓑寰呯殑鍑嗗垯锛夛紝鍥犺岃繖绉嶉攣涔熷彨Spin Lock锛屽湪鐭殏鐨勭瓑寰呬腑瀹冨彲浠ユ彁鍗囨ц兘锛屽洜涓哄彲浠ュ噺灏戠嚎绋嬬殑鍒囨崲锛宑oncurrent鍖呬腑鐨凙tomic澶ч儴鍒嗛兘閲囩敤杩欑鏈哄埗瀹炵幇锛岀劧鑰屽鏋滈渶瑕侀暱鏃堕棿鐨勭瓑寰咃紝“蹇欑瓑”浼氬崰鐢ㄤ笉蹇呰鐨凜PU鏃堕棿锛屼粠鑰屾ц兘浼氬彉鐨勫緢宸紝杩欎釜鏃跺欏氨闇瑕佸皢娌℃湁鎷垮埌閿佺殑绾跨▼鏀惧埌绛夊緟鍒楄〃涓紝杩欑鏂瑰紡鍦ㄦ搷浣滅郴缁熶腑涔熷彨璁板綍鍨嬩俊鍙烽噺锛屽畠閬靛惊浜嗚鏉冪瓑寰呭噯鍒欙紙褰撳墠娌℃湁瀹炵幇鏈夐檺绛夊緟鍑嗗垯锛夈傚湪JDK6浠ュ悗鎻愪緵浜哃ockSupport.park()/LockSupport.unpark()鎿嶄綔锛屽彲浠ュ皢褰撳墠绾跨▼鏀惧叆涓涓瓑寰呭垪琛ㄦ垨灏嗕竴涓嚎绋嬩粠杩欎釜绛夊緟鍒楄〃涓敜閱掋傜劧鑰岃繖涓猵ark/unpark鐨勭瓑寰呭垪琛ㄦ槸涓涓叏灞鐨勭瓑寰呭垪琛紝鍦╱npartk鐨勬椂鍊欒繕鏄渶瑕佹彁渚涢渶瑕佸敜閱掔殑Thread瀵硅薄锛屽洜鑰屾垜浠渶瑕佺淮鎶よ嚜宸辩殑绛夊緟鍒楄〃锛屼絾鏄鏋滄垜浠彲浠ョ敤JDK鎻愪緵鐨勫伐鍏风被ConcurrentLinkedQueue锛屽氨闈炲父瀹规槗瀹炵幇锛屽LockSupport鏂囨。涓粰鍑烘潵鐨?a >浠g爜浜嬩緥锛?br />

class FIFOMutex {
   
private final AtomicBoolean locked = new AtomicBoolean(false);
   
private final Queue<Thread> waiters = new ConcurrentLinkedQueue<Thread>();

   
public void lock() {
     
boolean wasInterrupted = false;
     Thread current 
= Thread.currentThread();
     waiters.add(current);

     
// Block while not first in queue or cannot acquire lock
     while (waiters.peek() != current || !locked.compareAndSet(falsetrue)) {
        LockSupport.park(
this);
        
if (Thread.interrupted()) // ignore interrupts while waiting
          wasInterrupted = true;
     }

     waiters.remove();
     
if (wasInterrupted)          // reassert interrupt status on exit
        current.interrupt();
   }

   
public void unlock() {
     locked.set(
false);
     LockSupport.unpark(waiters.peek());
   }
 }

鍦ㄨ浠g爜浜嬩緥涓紝鏈変竴涓嚎绋嬬瓑寰呴槦鍒楀拰閿佹爣璁板瓧娈碉紝姣忔璋冪敤lock鏃跺厛灏嗗綋鍓嶇嚎绋嬫斁鍏ヨ繖涓瓑寰呴槦鍒椾腑锛岀劧鍚庢嬁鍑洪槦鍒楀ご绾跨▼瀵硅薄锛屽鏋滆绾跨▼瀵硅薄姝eソ鏄綋鍓嶇嚎绋嬶紝骞朵笖鎴愬姛 浣跨敤CAS鏂瑰紡璁剧疆locked瀛楁锛堣繖閲岄渶瑕佷袱涓悓鏃舵弧瓒筹紝鍥犱负鍙兘鍑虹幇涓涓嚎绋嬪凡缁忎粠闃熷垪涓Щ闄や簡浣嗚繕娌℃湁unlock锛屾鏃跺彟涓涓嚎绋嬭皟鐢╨ock鏂规硶锛屾鏃堕槦鍒楀ご鐨勭嚎绋嬪氨鏄浜屼釜绾跨▼锛岀劧鑰岀敱浜庣涓涓嚎绋嬭繕娌℃湁unlock鎴栬呮鍦╱nlock锛屽洜鑰岄渶瑕佷娇鐢–AS鍘熷瓙鎿嶄綔鏉ュ垽鏂槸鍚﹁park锛夛紝琛ㄧず璇ョ嚎绋嬬珵浜夋垚鍔燂紝鑾峰緱閿侊紝鍚﹀垯灏嗗綋鍓嶇嚎绋媝ark锛岃繖閲屼箣鎵浠ヨ鏀惧湪 while寰幆涓紝鍥犱负park鎿嶄綔鍙兘鏃犵悊鐢辫繑鍥?spuriously)锛屽鏂囨。涓粰鍑虹殑鎻忚堪锛?br />

LockSupport.park()
public static void park(Object blocker)
Disables the current thread for thread scheduling purposes unless the permit is available.

If the permit is available then it is consumed and the call returns immediately; otherwise the current thread becomes disabled for thread scheduling purposes and lies dormant until one of three things happens:

  • Some other thread invokes unpark with the current thread as the target; or
  • Some other thread interrupts the current thread; or
  • The call spuriously (that is, for no reason) returns.

This method does not report which of these caused the method to return. Callers should re-check the conditions which caused the thread to park in the first place. Callers may also determine, for example, the interrupt status of the thread upon return.

Parameters:
blocker - the synchronization object responsible for this thread parking
Since:
1.6
鎴戝湪瀹炵幇鑷繁鐨勭被鏃跺氨琚繖涓?#8220;鏃犵悊鐢辫繑鍥?#8221;鍧戜簡濂戒箙銆傚浜庡凡缁忚幏寰楅攣鐨勭嚎绋嬶紝灏嗚绾跨▼浠庣瓑寰呴槦鍒椾腑绉婚櫎锛岃繖閲岀敱浜嶤oncurrentLinkedQueue鏄嚎绋嬪畨鍏ㄧ殑锛屽洜鑰岃兘淇濊瘉姣忔閮芥槸闃熷垪澶寸殑绾跨▼寰楀埌閿侊紝鍥犺屽湪寰楀埌閿佸寵灏嗛槦鍒楀ご绉婚櫎銆倁nlock閫昏緫姣旇緝绠鍗曪紝鍙渶瑕佸皢locked瀛楁鎵撳紑锛堣缃负false锛夛紝鍞ら啋锛坲npark锛夐槦鍒楀ご鐨勭嚎绋嬪嵆鍙紝鐒跺悗璇ョ嚎绋嬩細缁х画鍦╨ock鏂规硶鐨剋hile寰幆涓户缁珵浜塽nlocked瀛楁锛屽苟灏嗗畠鑷繁浠庣嚎绋嬮槦鍒椾腑绉婚櫎琛ㄧず鑾峰緱閿佹垚鍔熴傚綋鐒跺畨鍏ㄨ捣瑙侊紝鏈濂藉湪unlock涓姞鍏ヤ竴浜涢獙璇侀昏緫锛屽瑙i攣鐨勭嚎绋嬪拰鍔犻攣鐨勭嚎绋嬮渶瑕佺浉鍚屻?br />
鐒惰屾湰鏂囩殑鐩殑鏄嚜宸卞疄鐜颁竴涓狶ock瀵硅薄锛屽嵆鍙娇鐢ㄤ竴浜涘熀鏈殑鎿嶄綔锛岃屼笉浣跨敤JDK鎻愪緵鐨凙tomic绫诲拰ConcurrentLinkedQueue銆傜被浼肩殑棣栧厛鎴戜滑涔熼渶瑕佷竴涓槦鍒楀瓨鏀剧瓑寰呯嚎绋嬮槦鍒楋紙鍏钩璧疯锛屼娇鐢ㄥ厛杩涘厛鍑洪槦鍒楋級锛屽洜鑰屽厛瀹氫箟涓涓狽ode瀵硅薄鐢ㄤ互鏋勬垚杩欎釜闃熷垪锛?br />

 

    protected static class Node {
        
volatile Thread owner;
        
volatile Node prev;
        
volatile Node next;
        
        
public Node(Thread owner) {
            
this.owner = owner;
            
this.state = INIT;
        }
        
        
public Node() {
            
this(Thread.currentThread());
        }
    }

绠鍗曡捣瑙侊紝闃熷垪澶存槸涓涓捣鐐圭殑placeholder锛屾瘡涓皟鐢╨ock鐨勭嚎绋嬮兘鍏堝皢鑷繁绔炰簤鏀惧叆杩欎釜闃熷垪灏撅紝姣忎釜闃熷垪澶村悗涓涓嚎绋嬶紙Node锛夊嵆鏄幏寰楅攣鐨勭嚎绋嬶紝鎵浠ユ垜浠渶瑕佹湁head Node瀛楁鐢ㄤ互蹇熻幏鍙栭槦鍒楀ご鐨勫悗涓涓狽ode锛岃宼ail Node瀛楁鐢ㄦ潵蹇熸彃鍏ユ柊鐨凬ode锛屾墍浠ュ叧閿湪浜庡浣曠嚎绋嬪畨鍏ㄧ殑鏋勫缓杩欎釜闃熷垪锛屾柟娉曡繕鏄竴鏍风殑锛屼娇鐢–AS鎿嶄綔锛屽嵆CAS鏂规硶灏嗚嚜宸辫缃垚tail鍊硷紝鐒跺悗閲嶆柊鏋勫缓杩欎釜鍒楄〃锛?br />

    protected boolean enqueue(Node node) {
        
while (true) {
            
final Node preTail = tail;
            node.prev 
= preTail;
            
if (compareAndSetTail(preTail, node)) {
                preTail.next 
= node;
                
return node.prev == head;
            }
        }
    }

鍦ㄥ綋鍓嶇嚎绋婲ode浠ョ嚎绋嬪畨鍏ㄧ殑鏂瑰紡鏀惧叆杩欎釜闃熷垪鍚庯紝lock瀹炵幇鐩稿灏辨瘮杈冪畝鍗曚簡锛屽鏋滃綋鍓峃ode鏄殑鍓嶉┍鏄痟ead锛岃绾跨▼鑾峰緱閿侊紝鍚﹀垯park褰撳墠绾跨▼锛屽鐞唒ark鏃犵悊鐢辫繑鍥炵殑闂锛屽洜鑰屽皢park鏀惧叆while寰幆涓紙璇ュ疄鐜版槸涓涓笉鍙噸鍏ョ殑瀹炵幇锛夛細

    public void lock() {
        
// Put the latest node to a queue first, then check if the it is the first node
        
// this way, the list is the only shared resource to deal with
        Node node = new Node();
        
if (enqueue(node)) {
            current 
= node.owner;
        } 
else {
            
while (node.prev != head) {
                LockSupport.park(
this); // This may return "spuriously"!!, so put it to while
            }

            current 
= node.owner;
        }
    }

unlock鐨勫疄鐜伴渶瑕佽冭檻澶氱鎯呭喌锛屽鏋滃綋鍓峃ode(head.next)鏈夊悗椹憋紝閭d箞鐩存帴unpark璇ュ悗椹卞嵆鍙紱濡傛灉娌℃湁锛岃〃绀哄綋鍓嶅凡缁忔病鏈夊叾浠栫嚎绋嬪湪绛夊緟闃熷垪涓紝鐒惰屽湪杩欎釜鍒ゆ柇杩囩▼涓彲鑳戒細鏈夊叾浠栫嚎绋嬭繘鍏ワ紝鍥犺岄渶瑕佺敤CAS鐨勬柟寮忚缃畉ail锛屽鏋滆缃け璐ワ紝琛ㄧず姝ゆ椂鏈夊叾浠栫嚎绋嬭繘鍏ワ紝鍥犺岄渶瑕佸皢璇ユ柊杩涘叆鐨勭嚎绋媢npark浠庤岃鏂拌繘鍏ョ殑绾跨▼鍦ㄨ皟鐢╬ark鍚庡彲浠ョ珛鍗宠繑鍥烇紙杩欓噷鐨凜AS鍜宔nqueue鐨凜AS閮芥槸瀵箃ail鎿嶄綔锛屽洜鑰岃兘淇濊瘉鐘舵佷竴鑷达級锛?br />

    public void unlock() {
        Node curNode 
= unlockValidate();
        Node next 
= curNode.next;
        
if (next != null) {
           
head.next = next;
            next.prev 
= head;
            LockSupport.unpark(next.owner);
        } 
else {
            
if (!compareAndSetTail(curNode, head)) {
               
while (curNode.next == null) { } // Wait until the next available
                // Another node queued during the time, so we have to unlock that, or else, this node can never unparked
                unlock();
            } 
else {
               
compareAndSetNext(head, curNode, null); // Still use CAS here as the head.next may already been changed
            }
        }
    }

鍏蜂綋鐨勪唬鐮佸拰娴嬭瘯绫诲彲浠ュ弬鑰冩煡鐪?a >杩欓噷銆?br />


鍏跺疄鐩村埌鑷繁鍐欏畬杩欎釜绫诲悗鎵嶇洿鍒拌呭叾瀹炶繖鏄竴涓狹CS閿佺殑鍙樼锛屽洜鑰岃繖涓疄鐜版瘡涓嚎绋媝ark鍦ㄨ嚜韬搴旂殑node涓婏紝鑰岀敱鍓嶄竴涓嚎绋媢npark瀹冿紱鑰孉bstractQueuedSynchronizer鏄疌LH閿侊紝鍥犱负瀹冪殑park鐢卞墠椹辩姸鎬佸喅瀹氾紝铏界劧瀹冧篃鏄敱鍓嶄竴涓嚎绋媢npark瀹冦傚叿浣撳彲浠ュ弬鑰?a >杩欓噷銆?/p>

]]>
  • 苹果-热门标签-华商生活 2019-03-10
  • “奥运新秀”亮相上海 亚帆联杯等你来看 2019-03-07
  • 蔡奇就历史文化遗产保护调研:把燕都金中都建设保护好 2019-02-23
  • 红曲美的绿盒子,绿色生活大不同 2019-02-23
  • 理发迎接世界杯 梅西粉丝将偶像笑脸“抛在脑后” 2018-11-26
  • 我省各类创业孵化载体累计“毕业”企业3054家 2018-11-26
  • 鸿山慈善会厦门佛事展举行义诊活动 赢得群众交口称赞 2018-11-24
  • 特色小镇里的税收服务 定海国地税多措并举提升效能 2018-11-24
  • 端午小长假我市旅游总收入同比增20.5% 2018-08-12
  • [微笑]其实很简单就能破这个局:立法禁止通过房地产二次交易获利,炒房就会被杜绝,炒房一旦被杜绝,房价就会受正常供需关系影响波动在合理范围内。 2018-08-12