Exploiting Ceph: Flags for Managing OSD Natural States







Ceph , . , : noup, nodown, noin, noout. , « OSD» .







, « ». Ceph «».







OSD Ceph



, OSD Ceph.







.













OSD . OSD, — OSD.







up down. — in out.







up down — OSD. OSD up, — down.







in out — OSD, , OSD. OSD in, — out.







up/down in/out.







OSD up, , . , OSD down, .







OSD in, , « »: , OSD . . .







OSD out — .







. 9 OSD — up, in.







OSD 0.







systemctl stop ceph-osd@0
      
      





:







ceph -s
      
      





up 8 OSD, in - 9.













OSD down.







ceph osd tree down
      
      





down OSD.













down, , , OSD in.







— 20 . placement groups down OSD degraded, . in, . 20 OSD out.







, up/down in/out .







-



up, down, in, out — , Ceph . .







, (, OSD), .







OSD — noup/nodown noin/noout.







:







ceph osd set < >
      
      





:







ceph osd unset < >
      
      





.







noup



OSD down.







noup — OSD up ( , ).







ceph osd set noup
      
      





:













OSD.







systemctl start ceph-osd@0
      
      





- OSD up. , .













: OSD , OSD, , «noup». OSD , , up.







, , OSD down - OSD, up, , .







, noup. , OSD , noin ( ). OSD ( ), OSD . , , , , , .

.







OSD down.













- 9 OSD in.













OSD out :







ceph osd out 0
      
      





in :













, Ceph 20 .







Ceph . recovery io — placement groups OSD.













noup.







ceph osd unset noup
      
      





:







ceph osd tree up
      
      





OSD up.



















OSD out.







ceph osd tree out
      
      











: OSD up, out. .













.







systemctl stop ceph-osd@0
      
      





OSD .













in, : «- , , ».







ceph osd in 0
      
      





, OSD (, - ), .













.







noin



, - - OSD down ( ).







systemctl stop ceph-osd@0
      
      





Ceph out ( , ).







ceph osd out 0
      
      





, 8 up, 8 in.













, OSD, degraded, .













, , .







, , OSD (backfilling) placement group.







noin OSD. OSD out, , .







:







ceph osd set noin
      
      





.







, OSD out, placement groups degraded. placement group placement groups .













, out, — misplaced.













, , placement group degraded.







noin:







ceph osd unset noin
      
      





! , OSD, out, in :







ceph osd in 0
      
      





in, up .

OSD up, , .







noout



noout .







, OSD , Ceph rebalancing, , , OSD . , maintenance.







, .







nodown OSD



nodown OSD down. OSD up, : «, OSD , ».







nodown . , OSD down, . , ( ).







, nodown . « OSD» (flapping OSD's).







, , OSD down. : OSD down , - OSD , , down.







OSD , , OSD down. , OSD down OSD.







OSD OSD. , OSD . OSD . ( — ) , OSD , down. , OSD down, .







.







- Ceph OSD , . , OSD OSD, . - , - , OSD RocksDB , - , CPU.







, OSD , — , , OSD . , OSD, primary OSD.







OSD : , down — . . OSD down, , … , .







Ceph- «». . , down.







ceph osd set nodown
      
      





OSD .







, , OSD «» heart beat, , .







: nodown , , .







Ceph: nobackfill, norecover, norebalance pause.








All Articles