2018年11月22日 星期四

TODO: NAND

Source https://hk.saowen.com/a/40df7145abfef183a8ba6ee8551e66514b1e2188d7c48e58d18a11e90cc81d70


説説NAND FLASH以及相關ECC校驗方法


Flash名稱的由來,Flash的擦除操作是以block塊為單位的,與此相對應的是其他很多存儲設備,是以bit位為最小讀取/寫入的單位,Flash是一次性地擦除整個塊:在發送一個擦除命令後,一次性地將一個block,常見的塊的大小是128KB/256KB,全部擦除為1,也就是裏面的內容全部都是0xFF了,由於是一下子就擦除了,相對來説,擦除用的時間很短,可以用一閃而過來形容,所以,叫做Flash Memory。所以一般將Flash翻譯為 (快速)閃存。

 

NAND Flash 在嵌入式系統中有着廣泛的應用,負載平均和壞塊管理是與之相關的兩個核心議題。Uboot 和 Linux 系統對 NAND 的操作都封裝了對這兩個問題的處理方法。 本文首先講述Nandflash基礎知識,然後介紹現有的幾類壞塊管理(BBM)方法,通過分析典型嵌入式系統的 NAND 存儲表,指出了輕量級管理方法的優勢所在,分析了當前廣泛使用的輕量級管理方法,指出其缺陷所在並詳細説明了改進方法。
基礎知識 
Flash的硬件實現機制
Flash的內部存儲是MOSFET,裏面有個懸浮門(Floating Gate),是真正存儲數據的單元。
在Flash之前,紫外線可擦除(uv-erasable)的EPROM,就已經採用了Floating Gate存儲數據這一技術了。
典型的Flash內存物理結構
數據在Flash內存單元中是以電荷(electrical charge) 形式存儲的。存儲電荷的多少,取決於圖中的外部門(external gate)所被施加的電壓,其控制了是向存儲單元中衝入電荷還是使其釋放電荷。而數據的表示,以所存儲的電荷的電壓是否超過一個特定的閾值Vth來表示,因此,Flash的存儲單元的默認值,不是0(其他常見的存儲設備,比如硬盤燈,默認值為0),而是1,而如果將電荷釋放掉,電壓降低到一定程度,表述數字0。

NandFlash的簡介 
Nand flash成本相對低,説白了就是便宜,缺點是使用中數據讀寫容易出錯,所以一般都需要有對應的軟件或者硬件的數據校驗算法,統稱為ECC。但優點是,相對來説容量比較大,現在常見的Nand Flash都是1GB,2GB,更大的8GB的都有了,相對來説,價格便宜,因此適合用來存儲大量的數據。其在嵌入式系統中的作用,相當於PC上的硬盤,用於存儲大量數據。
SLC和MLC 
Nand Flash按照內部存儲數據單元的電壓的不同層次,也就是單個內存單元中,是存儲1位數據,還是多位數據,可以分為SLC和MLC。那麼軟件如何識別系統上使用過的SLC還是MLC呢?
Nand Flash設計中,有個命令叫做Read ID,讀取ID,讀取好幾個字節,一般最少是4個,新的芯片,支持5個甚至更多,從這些字節中,可以解析出很多相關的信息,比如此Nand Flash內部是幾個芯片(chip)所組成的,每個chip包含了幾片(Plane),每一片中的頁大小,塊大小,等等。在這些信息中,其中有一個,就是識別此flash是SLC還是MLC。 
 oob / Redundant Area / Spare Area
每一個頁,對應還有一塊區域,叫做空閒區域(spare area)/宂餘區域(redundant area),而Linux系統中,一般叫做OOB(Out Of Band),這個區域,是最初基於Nand Flash的硬件特性:數據在讀寫時候相對容易錯誤,所以為了保證數據的正確性,必須要有對應的檢測和糾錯機制,此機制被叫做EDC(Error Detection Code)/ECC(Error Code Correction, 或者 Error Checking and Correcting),所以設計了多餘的區域,用於放置數據的校驗值。
Oob的讀寫操作,一般是隨着頁的操作一起完成的,即讀寫頁的時候,對應地就讀寫了oob。
關於oob具體用途,總結起來有:
  1. 標記是否是壞快
  2. 存儲ECC數據
  3. 存儲一些和文檔系統相關的數據。如jffs2就會用到這些空間存儲一些特定信息,而yaffs2文檔系統,會在oob中,存放很多和自己文檔系統相關的信息。

Bad Block Management壞塊管理
Nand Flash由於其物理特性,只有有限的擦寫次數,超過那個次數,基本上就是壞了。在使用過程中,有些Nand Flash的block會出現被用壞了,當發現了,要及時將此block標註為壞塊,不再使用。於此相關的管理工作,屬於Nand Flash的壞塊管理的一部分工作。
Wear-Leveling負載平衡
Nand Flash的block管理,還包括負載平衡。
正是由於Nand Flash的block,都是有一定壽命限制的,所以如果你每次都往同一個block擦除然後寫入數據,那麼那個block就很容易被用壞了,所以我們要去管理一下,將這麼多次的對同一個block的操作,平均分佈到其他一些block上面,使得在block的使用上,相對較平均,這樣相對來説,可以更能充分利用Nand Flash。
 ECC錯誤校驗碼
Nand Flash物理特性上使得其數據讀寫過程中會發生一定機率的錯誤,所以要有個對應的錯誤檢測和糾正的機制,於是才有此ECC,用於數據錯誤的檢測與糾正。Nand Flash的ECC,常見的算法有海明碼和BCH,這類算法的實現,可以是軟件也可以是硬件。不同系統,根據自己的需求,採用對應的軟件或者是硬件。
相對來説,硬件實現這類ECC算法,肯定要比軟件速度要快,但是多加了對應的硬件部分,所以成本相對要高些。如果系統對於性能要求不是很高,那麼可以採用軟件實現這類ECC算法,但是由於增加了數據讀取和寫入前後要做的數據錯誤檢測和糾錯,所以性能相對要降低一些,即Nand Flash的讀取和寫入速度相對會有所影響。
其中,Linux中的軟件實現ECC算法,即NAND_ECC_SOFT模式,就是用的對應的海明碼。
而對於目前常見的MLC的Nand Flash來説,由於容量比較大,動輒2GB,4GB,8GB等,常用BCH算法。BCH算法,相對來説,算法比較複雜。
筆者由於水平有限,目前仍未完全搞懂BCH算法的原理。
BCH算法,通常是由對應的Nand Flash的Controller中,包含對應的硬件BCH ECC模塊,實現了BCH算法,而作為軟件方面,需要在讀取數據後,寫入數據之前,分別操作對應BCH相關的寄存器,設置成BCH模式,然後讀取對應的BCH狀態寄存器,得知是否有錯誤,和生成的BCH校驗碼,用於寫入。
其具體代碼是如何操作這些寄存器的,由於是和具體的硬件,具體的nand flash的controller不同而不同,無法用同一的代碼。如果你是nand flash驅動開發者,自然會得到對應的起nand flash的controller部分的datasheet,按照手冊説明,去操作即可。
不過,額外説明一下的是,關於BCH算法,往往是要從專門的做軟件算法的廠家購買的,但是Micron之前在網上放出一個免費版本的BCH算法。
位反轉
Nand Flash的位反轉現象,主要是由以下一些原因/效應所導致:
  1. 漂移效應(Drifting Effects)
    漂移效應指的是,Nand Flash中cell的電壓值,慢慢地變了,變的和原始值不一樣了。
  2. 編程干擾所產生的錯誤(Program-Disturb Errors)
    此現象有時候也叫做,過度編程效應(over-program effect)。
    對於某個頁面的編程操作,即寫操作,引起非相關的其他的頁面的某個位跳變了。
  3. 讀操作干擾產生的錯誤(Read-Disturb Errors)
    此效應是,對一個頁進行數據讀取操作,卻使得對應的某個位的數據,產生了永久性的變化,即Nand Flash上的該位的值變了。

對應位反轉的類型,Nand Flash位反轉的類型和解決辦法,有兩種:

  1. 一種是nand flash物理上的數據存儲的單元上的數據,是正確的,只是在讀取此數據出來的數據中的某位,發生變化,出現了位反轉,即讀取出來的數據中,某位錯了,本來是0變成1,或者本來是1變成0了。此處可以成為軟件上位反轉。此數據位的錯誤,當然可以通過一定的校驗算法檢測並糾正。
  2. 另外一種,就是nand flash中的物理存儲單元中,對應的某個位,物理上發生了變化,原來是1的,變成了0,或原來是0的,變成了1,發生了物理上的位的數據變化。此處可以成為硬件上的位反轉。此錯誤,由於是物理上發生的,雖然讀取出來的數據的錯誤,可以通過軟件或硬件去檢測並糾正過來,但是物理上真正發生的位的變化,則沒辦法改變了。不過個人理解,好像也是可以通過擦除Erase整個數據塊Block的方式去擦除此錯誤,不過在之後的Nand Flash的使用過程中,估計此位還是很可能繼續發生同樣的硬件的位反轉的錯誤。
以上兩種類型的位反轉,其實對於從Nand Flash讀取出來的數據來説,解決其中的錯誤的位的方法,都是一樣的,即通過一定的校驗算法,常稱為ECC,去檢測出來,或檢測並糾正錯誤。
如果只是單獨檢測錯誤,那麼如果發現數據有誤,那麼再重新讀取一次即可。
實際中更多的做法是,ECC校驗發現有錯誤,會有對應的算法去找出哪位錯誤並且糾正過來。
其中對錯誤的檢測和糾正,具體的實現方式,有軟件算法,也有硬件實現,即硬件Nand Flash的控制器controller本身包含對應的硬件模塊以實現數據的校驗和糾錯的。

Nand Flash的一些typical特性

  1. 頁擦除時間是200us,有些慢的有800us
  2. 塊擦除時間是1.5ms
  3. 頁數據讀取到數據寄存器的時間一般是20us
  4. 串行訪問(Serial access)讀取一個數據的時間是25ns,而一些舊的Nand Flash是30ns,甚至是50ns
  5. 輸入輸出端口是地址和數據以及命令一起multiplex複用的
  6. Nand Flash的編程/擦除的壽命:即,最多允許10萬次的編程/擦除,達到和接近於之前常見的Nor Flash,幾乎是同樣的使用壽命了。
  7. 封裝形式:48引腳的TSOP1封裝 或 52引腳的ULGA封裝

Nand Flash控制器與Nand Flash芯片

我們寫驅動,是寫Nand Flash 控制器的驅動,而不是Nand Flash 芯片的驅動,因為獨立的Nand Flash芯片,一般來説,是很少直接拿來用的,多數都是硬件上有對應的硬件的Nand Flash的控制器,去操作和控制Nand Flash,包括提供時鐘信號,提供硬件ECC校驗等等功能,我們所寫的驅動軟件,是去操作Nand Flash的控制器
然後由控制器去操作Nand Flash芯片,實現我們所要的功能。
由於Nand Flash讀取和編程操作來説,一般最小單位是頁,所以Nand Flash在硬件設計時候,就考慮到這一特性,對於每一片(Plane),都有一個對應的區域專門用於存放,將要寫入到物理存儲單元中去的或者剛從存儲單元中讀取出來的,一頁的數據,這個數據緩存區,本質上就是一個緩存buffer,但是隻是此處datasheet裏面把其叫做頁寄存器page register而已,實際將其理解為頁緩存,更貼切原意。
而正是因為有些人不瞭解此內部結構,才容易產生之前遇到的某人的誤解,以為內存裏面的數據,通過Nand Flash的FIFO,寫入到Nand Flash裏面去,就以為立刻實現了實際數據寫入到物理存儲單元中了,而實際上只是寫到了這個頁緩存中,只有當你再發送了對應的編程第二階段的確認命令,即0x10,之後,實際的編程動作才開始,才開始把頁緩存中的數據,一點點寫到物理存儲單元中去。

壞塊的標記

具體標記的地方是,對於現在常見的頁大小為2K的Nand Flash,是塊中第一個頁的oob起始位置的第1個字節(舊的小頁面,pagesize是512B甚至256B的Nand Flash,壞塊標記是第6個字節),如果不是0xFF,就説明是壞塊。相對應的是,所有正常的塊,好的塊,裏面所有數據都是0xFF的。
對於壞塊的標記,本質上,也只是對應的flash上的某些字節的數據是非0xFF而已,所以,只要是數據,就是可以讀取和寫入的。也就意味着,可以寫入其他值,也就把這個壞塊標記信息破壞了。對於出廠時的壞塊,一般是不建議將標記好的信息擦除掉的。
uboot中有個命令是
nand scrub
就可以將塊中所有的內容都擦除了,包括壞塊標記,不論是出廠時的,還是後來使用過程中出現而新標記的。
nand erase
只擦除好的塊,對於已經標記壞塊的塊,不要輕易擦除掉,否則就很難區分哪些是出廠時就壞的,哪些是後來使用過程中用壞的了。

Uboot 的輕量級壞塊管理方法

NAND 壞塊管理都是基於壞塊表(BBT)的,通過這張表來標識系統中的所有壞塊。所以,不同的管理方法之間的差異可以通過以下幾個問題來找到答案。
  • 如何初始化和讀取壞塊表?
  • 產生新的壞塊時,如何標記並更新壞塊表?
  • 如何保存壞塊表?是否有保存時斷電保護機制?
  • 對 NAND 寫入數據時,如果當前塊是壞塊,如何找到可替換的好塊?
Uboot 是目前使用最為廣泛的 bootloader,它提供了兩種輕量級壞塊管理方法,可稱之為基本型和改進型。通過下表,我們可以看到兩者的差異。
雖然 uboot 的改進型壞塊管理方法的做了一些改進,但它仍然有三個主要的缺點。
  1. 出現壞塊,則將數據順序寫入下一個好塊。如果 NAND 中存放了多個軟件模塊,則每個模塊都需要預留一個較大的空間作為備用的好塊,這會浪費較多的 NAND 空間。通常,每個模塊預留的備用好塊數為 NAND 芯片所允許的最大壞塊數,該值因不同的芯片而有所不同,典型值為 20 或 80。假設 NAND 是大頁類型,總共有 N 個模塊,則總共需要預留的空間大小為 N*80*128KB。
  2. 讀取 BBT 時僅檢查簽名,沒有對 BBT 的數據做校驗。
  3. 沒有掉電保護機制。如果在保存 BBT 時斷電,BBT 將丟失。
針對現有管理方法的缺陷,本文提出了一種更加安全高效的管理方法,將從以下三個方面闡述其實現原理。

共用好塊池機制

首先,使用一個統一的備用好塊池,為所有存放在 NAND 中的模塊提供可替換的好塊。這樣,就不需要在每個模塊後面放置一個保留區,提高了 NAND 的空間利用率。
共用好塊池示意圖
共用好塊池示意圖
為了實現共用好塊池,需要創建一個從壞塊到好塊的映射,所以,除了 BBT 之外,還需定義一個替換表(SBT)。這樣一來,當讀第 i 個塊的數據時,如果發現 BBT 中記錄該塊為壞塊,就去 SBT 中查詢其替換塊;如果寫第 i 個塊出錯,需要在 BBT 中標記該塊為壞塊,同時從好塊池中獲取一個新的好塊,假設其序號為 j,然後將此好塊的序號 j 寫入 SBT 中的第 i 個字節,而且 SBT 的第 j 個字節寫序號 i。SBT 中的這種雙向映射可確保數據的可靠性。此外,好塊池中的塊也有可能成為壞塊,如果掃描時發現是壞塊,則將 SBT 中的對應位置標記為 0x00,如果是在寫的過程中出錯,則除了在 SBT 對應位置標記 0x00 之外,還要更新雙向映射數據。
 BBT/SBT 映射示意圖

安全的 BBT/SBT 數據校驗機制

傳統方法僅檢查 BBT 所在塊的簽名,將讀到的前幾個字節和一個特徵字符串進行比較,如果一致,就認為當前塊的數據為 BBT,然後讀取接下來的 BBT 數據,但並不對 BBT 的數據做校驗。如果 BBT 保存在 NAND 中,數據的有效性是可以得到驗證的,因為 NAND 控制器或驅動一般都會對數據做 ECC 校驗。但是,大多數控制器使用的 ECC 算法也僅僅能糾正一個 bit、發現 2 兩個 bit 的錯誤。如果 BBT 保存在其他的沒有 ECC 校驗機制的存儲體中,比如 NOR Flash,沒有對 BBT 的數據進行校驗顯然是不安全的。
為了更加可靠和靈活地驗證 BBT/SBT 數據,定義下面這個結構體來描述 BBM 信息。
BBM 頭信息
typedef struct {
UINT8     acSignature[4];/* BBM 簽名 */
UINT32    ulBBToffset;/* BBT 偏移 */
UINT32    ulSBToffset;/* SBT 偏移 */
UINT16    usBlockNum;/* BBM 管理的 block 數目 */    
UINT16    usSBTstart;/* SBT 所在位置的起始 block 序號 */
UINT16    usSBtop;/* SBT top block */
UINT16    usSBnum;/* SBT number */
UINT32    ulBBTcrc;/* BBT 數據 CRC 校驗碼 */
UINT32    ulSBTcrc;/* SBT 數據 CRC 校驗碼 */
UINT32    ulHeadcrc;/* BBM 頭信息 CRC 校驗碼 */
} BBM_HEAD
BBT/SBT 的保存形式
BBT/SBT 的保存形式
使用三重 CRC 校驗機制,無論 BBT 保存在哪種存儲體中,都可以更加嚴格地驗證數據的有效性。

安全的掉電保存機制

傳統的方法僅保存一份 BBT 數據,如果在寫 BBT 時系統掉電,則 BBT 丟失,系統將可能無法正常啟動或工作。為安全起見,本文所述方法將同時保留三個備份,如果在寫某個備份時掉電,則還有兩個完好的備份。最壞的情況是,如果在寫第一個備份時掉電,則當前最新的一個壞塊信息丟失。
讀取壞塊表時,順序讀取三個備份,如果發現三個備份的數據不一致,用記錄的壞塊數最多的備份為當前的有效備份,同時立刻更新另外兩備份。

總結

本文介紹了NandFlash基礎知識和幾類 NAND 壞塊管理方法,指出了 uboot 的輕量級管理方法的缺陷,提出了一種改進的方法,提高了 NAND 的利用率及壞塊管理的安全性,可對嵌入式開發起到有很好的借鑑作用。

ECC定義

ECC校驗是一種內存糾錯原理,它是比較先進的內存錯誤檢查和更正的手段。ECC內存即糾錯內存,簡單的説,其具有發現錯誤,糾正錯誤的功能,一般多應用在高檔台式電腦/服務器及圖形工作站上,這將使整個電腦系統在工作時更趨於安全穩定。

技術原理

內存是一種電子器件,在其工作過程中難免會出現錯誤,而對於穩定性要求高的用户來説,內存錯誤可能會引起致命性的問題。內存錯誤根據其原因還可分為硬錯誤和軟錯誤。硬件錯誤是由於硬件的損害或缺陷造成的,因此數據總是不正確,此類錯誤是無法糾正的;軟錯誤是隨機出現的,例如在內存附近突然出現電子干擾等因素都可能造成內存軟錯誤的發生。
為了能檢測和糾正內存軟錯誤,在ECC技術出現之前,首先出現的是內存“奇偶校驗(Parity)”。內存中最小的單位是比特,也稱為“位(bit)”,位有隻有兩種狀態分別以1和0來標示,每8個連續的比特叫做一個字節(byte)。不帶奇偶校驗的內存每個字節只有8位,如果其某一位存儲了錯誤的值,就會導致其存儲的相應數據發生變化,進而導致應用進程發生錯誤。而奇偶校驗就是在每一字節(8位)之外又增加了一位作為錯誤檢測位。在某字節中存儲數據之後,在其8個位上存儲的數據是固定的,因為位只能有兩種狀態1或0,假設存儲的數據用位標示為1、1、1、0、0、1、0、1,那麼把每個位相加(1+1+1+0+0+1+0+1=5),結果是奇數。對於偶校驗,校驗位就定義為1,反之則為0;對於奇校驗,則相反。當CPU讀取存儲的數據時,它會再次把前8位中存儲的數據相加,計算結果是否與校驗位相一致。從而一定程度上能檢測出內存錯誤,奇偶校驗只能檢測出錯誤而無法對其進行修正,同時雖然雙位同時發生錯誤的概率相當低,奇偶校驗卻無法檢測出雙位錯誤。
通過上面的分析我們知道Parity內存是通過在原來數據位的基礎上增加一個數據位來檢查當前8位數據的正確性,但隨着數據位的增加Parity用來檢驗的數據位也成倍增加,就是説當數據位為16位時它需要增加2位用於檢查,當數據位為32位時則需增加4位,依此類推。特別是當數據量非常大時,數據出錯的機率也就越大,對於只能糾正簡單錯誤的奇偶檢驗的方法就顯得力不從心了,正是基於這樣一種情況,一種新的內存技術應允而生了,這就是ECC(錯誤檢查和糾正),這種技術也是在原來的數據位上外加校驗位來實現的。不同的是兩者增加的方法不一樣,這也就導致了兩者的主要功能不太一樣。它與Parity不同的是如果數據位是8位,則需要增加5位來進行ECC錯誤檢查和糾正,數據位每增加一倍,ECC只增加一位檢驗位,也就是説當數據位為16位時ECC位為6位,32位時ECC位為7位,數據位為64位時ECC位為8位,依此類推,數據位每增加一倍,ECC位只增加一位。總之,在內存中ECC能夠容許錯誤,並可以將錯誤更正,使系統得以持續正常的操作,不致因錯誤而中斷,且ECC具有自動更正的能力,可以將Parity無法檢查出來的錯誤位查出並將錯誤修正。

示例

ECC(Error Checking and Correcting,錯誤檢查和糾正)內存,它同樣也是在數據位上額外的位存儲一個用數據加密的代碼。當數據被寫入內存,相應的ECC代碼與此同時也被保存下來。當重新讀回剛才存儲的數據時,保存下來的ECC代碼就會和讀數據時產生的ECC代碼做比較。如果兩個代碼不相同,他們則會被解碼,以確定數據中的哪一位是不正確的。然後這一錯誤位會被拋棄,內存控制器則會釋放出正確的數據。被糾正的數據很少會被放回內存。假如相同的錯誤數據再次被讀出,則糾正過程再次被執行。重寫數據會增加處理過程的開銷,這樣則會導致系統性能的明顯降低。如果是隨機事件而非內存的缺點產生的錯誤,則這一內存地址的錯誤數據會被再次寫入的其他數據所取代。

相關推薦:

TODO: DRAM test

Referece:
     https://github.com/littlekernel/lk/tree/master/app
     memtest


Source http://benjr.tw/96460

Linux command – Memtester

 701 total views, 2 views today
之前用過 Memtest86+ http://benjr.tw/491 ,無法直接在 Linux 環境下直接來使用,Memtester 可以直接在 Linux 環境下執行,且可以從 apt-get 下載使用,官方網站 http://pyropus.ca/software/memtester/
root@ubuntu:~# apt-get install memtester
Reading package lists... Done
Building dependency tree      
Reading state information... Done
The following packages were automatically installed and are no longer required:
  linux-headers-4.4.0-31 linux-headers-4.4.0-31-generic
  linux-image-4.4.0-31-generic linux-image-extra-4.4.0-31-generic
Use 'apt autoremove' to remove them.
The following NEW packages will be installed:
  memtester
0 upgraded, 1 newly installed, 0 to remove and 8 not upgraded.
Need to get 16.2 kB of archives.
After this operation, 66.6 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu xenial/universe amd64 memtester amd64 4.3.0-3 [16.2 kB]
Fetched 16.2 kB in 1s (11.6 kB/s)                     
Selecting previously unselected package memtester.
(Reading database ... 242316 files and directories currently installed.)
Preparing to unpack .../memtester_4.3.0-3_amd64.deb ...
Unpacking memtester (4.3.0-3) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up memtester (4.3.0-3) ...
使用上也很簡單,只需要指定要測試的記憶體大小 (單位: B/K/M/G) 以及測試次數 (Loops) 即可.
Usage: memtester [-p physaddrbase [-d device]] [B|K|M|G] [loops]
我的系統是 Ubuntu16.04 1G 記憶體 的 VMware 虛擬機器.
root@ubuntu:~# memtester 200M 1
memtester version 4.3.0 (64-bit)
Copyright (C) 2001-2012 Charles Cazabon.
Licensed under the GNU General Public License version 2 (only).
 
pagesize is 4096
pagesizemask is 0xfffffffffffff000
want 200MB (209715200 bytes)
got  200MB (209715200 bytes), trying mlock ...locked.
Loop 1/1:
  Stuck Address       : ok        
  Random Value        : ok
  Compare XOR         : ok
  Compare SUB         : ok
  Compare MUL         : ok
  Compare DIV         : ok
  Compare OR          : ok
  Compare AND         : ok
  Sequential Increment: ok
  Solid Bits          : ok        
  Block Sequential    : ok        
  Checkerboard        : ok        
  Bit Spread          : ok        
  Bit Flip            : ok        
  Walking Ones        : ok        
  Walking Zeroes      : ok        
  8-bit Writes        : ok
  16-bit Writes       : ok
 
Done.
Memtester 主要會針對記憶體做以下的測試
  1. Stuck Address
  2. Random Value -test random value
  3. Compare XOR – test xor comparison
  4. Compare SUB – test sub comparison
  5. Compare MUL – test mul comparison
  6. Compare DIV – test div comparison
  7. Compare OR – test or comparison
  8. Compare AND – test and comparison
  9. Sequential Increment – test seqinc comparison
  10. Solid Bits – test solidbits comparison
  11. Block Sequential – test blockseq comparison
  12. Checkerboard – test checkerboard comparison
  13. Bit Spread – test bitspread comparison
  14. Bit Flip – test bitflip comparison
  15. Walking Ones – test walk bits 1 comparison
  16. Walking Zeroes – test_walk bits 0 comparison
  17. 8-bit Writes – test 8bit wide random
  18. 16-bit Writes -test 16bit wide random
memtester 是透過 malloc 函數來要求記憶體空間,所以在設定測試記憶體空間時沒有辦法把全部的記憶體都拿來測試.
root@ubuntu:~# free
              total        used        free      shared  buff/cache   available
Mem:         998312      158264      740632          80       99416      698696
Swap:       1046524      492832      553692
透過 free 可以看到,系統大概還有 700M 可以用,所以上限大概也就在這裡.
Linux 底下記憶體的使用區分下面幾種.
  • Total Memory 
    系統所有的記憶體容量.
  • Used Memory
    Total(998,312) – Free(740632) – Buffers/Cache(99416) 就是目前所有有在使用中的記憶體數量.
  • Free Memory
    尚未使用到的的記憶體.
  • shared Memory
    used by tmpfs , 允許 processes (程序) 透過 Shared memory 共享儲存於記憶體中的 common structures 和 Data.
  • Buffered Memory
    資料還來不及儲存到硬碟中暫儲在記憶體的資料.
  • Cached Memory
    資料已經由硬碟中讀取出來,提供給應用程式接下來使用,用以提高存取效能.
  • Available
    估計有多少記憶體可用於啟動新應用程式,無需透過 swapping
超過系統能給的時候 memtester 就不會繼續執行.
root@ubuntu:~# memtester 1G 1
memtester version 4.3.0 (64-bit)
Copyright (C) 2001-2012 Charles Cazabon.
Licensed under the GNU General Public License version 2 (only).
 
pagesize is 4096
pagesizemask is 0xfffffffffffff000
want 1024MB (1073741824 bytes)
got  1024MB (1073741824 bytes), trying mlock ...Killed
root@ubuntu:~# memtester 700M 1
memtester version 4.3.0 (64-bit)
Copyright (C) 2001-2012 Charles Cazabon.
Licensed under the GNU General Public License version 2 (only).
 
pagesize is 4096
pagesizemask is 0xfffffffffffff000
want 700MB (734003200 bytes)
got  700MB (734003200 bytes), trying mlock ...locked.
Loop 1/1:
  Stuck Address       : ok        
  Random Value        : ok
  Compare XOR         : ok
  Compare SUB         : ok
  Compare MUL         : ok
  Compare DIV         : ok
  Compare OR          : ok
  Compare AND         : ok
  Sequential Increment: ok
  Solid Bits          : ok        
  Block Sequential    : ok        
  Checkerboard        : ok        
  Bit Spread          : ok        
  Bit Flip            : setting  99Killed

2018年11月21日 星期三

DEVICE TREES, OVERLAYS, AND PARAMETERS

Source https://www.raspberrypi.org/documentation/configuration/device-tree.md

DEVICE TREES, OVERLAYS, AND PARAMETERS

Raspberry Pi's latest kernels and firmware, including Raspbian and NOOBS releases, now use a Device Tree (DT) to manage some resource allocation and module loading by default. This was implemented to alleviate the problem of multiple drivers contending for system resources, and to allow HAT modules to be auto-configured.
The current implementation is not a pure Device Tree system – there is still board support code that creates some platform devices – but the external interfaces (I2C, I2S, SPI), and the audio devices that use them, must now be instantiated using a Device Tree Blob (DTB) passed to the kernel by the loader (start.elf).
The main impact of using Device Tree is to change from everything on, relying on module blacklisting to manage contention, to everything off unless requested by the DTB. In order to continue to use external interfaces and the peripherals that attach to them, you will need to add some new settings to your config.txt. Full details are given in Part 3, but these are a few examples:
# Uncomment some or all of these lines to enable the optional hardware interfaces
#dtparam=i2c_arm=on
#dtparam=i2s=on
#dtparam=spi=on

# Uncomment one of these lines to enable an audio interface
#dtoverlay=hifiberry-amp
#dtoverlay=hifiberry-dac
#dtoverlay=hifiberry-dacplus
#dtoverlay=hifiberry-digi
#dtoverlay=iqaudio-dac
#dtoverlay=iqaudio-dacplus
#dtoverlay=audioinjector-wm8731-audio

# Uncomment this to enable the lirc-rpi module
#dtoverlay=lirc-rpi

# Uncomment this to override the defaults for the lirc-rpi module
#dtparam=gpio_out_pin=16
#dtparam=gpio_in_pin=17
#dtparam=gpio_in_pull=down

Device Trees

A Device Tree (DT) is a description of the hardware in a system. It should include the name of the base CPU, its memory configuration, and any peripherals (internal and external). A DT should not be used to describe the software, although by listing the hardware modules it does usually cause driver modules to be loaded. It helps to remember that DTs are supposed to be OS-neutral, so anything which is Linux-specific probably shouldn't be there.
A Device Tree represents the hardware configuration as a hierarchy of nodes. Each node may contain properties and subnodes. Properties are named arrays of bytes, which may contain strings, numbers (big-endian), arbitrary sequences of bytes, and any combination thereof. By analogy to a filesystem, nodes are directories and properties are files. The locations of nodes and properties within the tree can be described using a path, with slashes as separators and a single slash (/) to indicate the root.

1.1: BASIC DTS SYNTAX

Device Trees are usually written in a textual form known as Device Tree Source (DTS) and stored in files with a .dts suffix. DTS syntax is C-like, with braces for grouping and semicolons at the end of each line. Note that DTS requires semicolons after closing braces: think of C structs rather than functions. The compiled binary format is referred to as Flattened Device Tree (FDT) or Device Tree Blob (DTB), and is stored in .dtb files.
The following is a simple tree in the .dts format:
/dts-v1/;
/include/ "common.dtsi";

/ {
    node1 {
        a-string-property = "A string";
        a-string-list-property = "first string", "second string";
        a-byte-data-property = [0x01 0x23 0x34 0x56];
        cousin: child-node1 {
            first-child-property;
            second-child-property = <1>;
            a-string-property = "Hello, world";
        };
        child-node2 {
        };
    };
    node2 {
        an-empty-property;
        a-cell-property = <1 2="" 3="" 4="">; /* each number (cell) is a uint32 */
        child-node1 {
            my-cousin = <&cousin>;
        };
    };
};

/node2 {
    another-property-for-node2;
};
This tree contains:
  • a required header: /dts-v1/.
  • The inclusion of another DTS file, conventionally named *.dtsi and analogous to a .h header file in C - see An aside about /include/ below.
  • a single root node: /
  • a couple of child nodes: node1 and node2
  • some children for node1: child-node1 and child-node2
  • a label (cousin) and a reference to that label (&cousin): see Labels and References below.
  • several properties scattered through the tree
  • a repeated node (/node2) - see An aside about /include/ below.
Properties are simple key-value pairs where the value can either be empty or contain an arbitrary byte stream. While data types are not encoded in the data structure, there are a few fundamental data representations that can be expressed in a Device Tree source file.
Text strings (NUL-terminated) are indicated with double quotes:
string-property = "a string";
Cells are 32-bit unsigned integers delimited by angle brackets:
cell-property = <0xbeef 0xabcd1234="" 123="">;
Arbitrary byte data is delimited with square brackets, and entered in hex:
binary-property = [01 23 45 67 89 ab cd ef];
Data of differing representations can be concatenated using a comma:
mixed-property = "a string", [01 23 45 67], <0x12345678>;
Commas are also used to create lists of strings:
string-list = "red fish", "blue fish";

1.2: AN ASIDE ABOUT /INCLUDE/

The /include/ directive results in simple textual inclusion, much like C's #include directive, but a feature of the Device Tree compiler leads to different usage patterns. Given that nodes are named, potentially with absolute paths, it is possible for the same node to appear twice in a DTS file (and its inclusions). When this happens, the nodes and properties are combined, interleaving and overwriting properties as required (later values override earlier ones).
In the example above, the second appearance of /node2 causes a new property to be added to the original:
/node2 {
    an-empty-property;
    a-cell-property = <1 2="" 3="" 4="">; /* each number (cell) is a uint32 */
    another-property-for-node2;
    child-node1 {
        my-cousin = <&cousin>;
    };
};
It is thus possible for one .dtsi to overwrite, or provide defaults for, multiple places in a tree.

1.3: LABELS AND REFERENCES

It is often necessary for one part of the tree to refer to another, and there are four ways to do this:
  1. Path strings
    Paths should be self-explanatory, by analogy with a filesystem - /soc/i2s@7e203000 is the full path to the I2S device in BCM2835 and BCM2836. Note that although it is easy to construct a path to a property (for example, /soc/i2s@7e203000/status), the standard APIs don't do that; you first find a node, then choose properties of that node.
  2. phandles
    A phandle is a unique 32-bit integer assigned to a node in its phandleproperty. For historical reasons, you may also see a redundant, matching linux,phandle. phandles are numbered sequentially, starting from 1; 0 is not a valid phandle. They are usually allocated by the DT compiler when it encounters a reference to a node in an integer context, usually in the form of a label (see below). References to nodes using phandles are simply encoded as the corresponding integer (cell) values; there is no markup to indicate that they should be interpreted as phandles, as that is application-defined.
  3. Labels
    Just as a label in C gives a name to a place in the code, a DT label assigns a name to a node in the hierarchy. The compiler takes references to labels and converts them into paths when used in string context (&node) and phandles in integer context (<&node>); the original labels do not appear in the compiled output. Note that labels contain no structure; they are just tokens in a flat, global namespace.
  4. Aliases
    Aliases are similar to labels, except that they do appear in the FDT output as a form of index. They are stored as properties of the /aliases node, with each property mapping an alias name to a path string. Although the aliases node appears in the source, the path strings usually appear as references to labels (&node), rather then being written out in full. DT APIs that resolve a path string to a node typically look at the first character of the path, treating paths that do not start with a slash as aliases that must first be converted to a path using the /aliases table.

1.4: DEVICE TREE SEMANTICS

How to construct a Device Tree, and how best to use it to capture the configuration of some hardware, is a large and complex subject. There are many resources available, some of which are listed below, but several points deserve mentioning in this document:
compatible properties are the link between the hardware description and the driver software. When an OS encounters a node with a compatible property, it looks it up in its database of device drivers to find the best match. In Linux, this usually results in the driver module being automatically loaded, provided it has been appropriately labelled and not blacklisted.
The status property indicates whether a device is enabled or disabled. If the status is okokay or absent, then the device is enabled. Otherwise, status should be disabled, so that the device is disabled. It can be useful to place devices in a .dtsi file with the status set to disabled. A derived configuration can then include that .dtsi and set the status for the devices which are needed to okay.

Part 2: Device Tree overlays

A modern SoC (System on a Chip) is a very complicated device; a complete Device Tree could be hundreds of lines long. Taking that one step further and placing the SoC on a board with other components only makes matters worse. To keep that manageable, particularly if there are related devices that share components, it makes sense to put the common elements in .dtsi files, to be included from possibly multiple .dts files.
But when a system like Raspberry Pi supports optional plug-in accessories, such as HATs, the problem grows. Ultimately, each possible configuration requires a Device Tree to describe it, but once you factor in different base hardware (models A, B, A+, and B+) and gadgets only requiring the use of a few GPIO pins that can coexist, the number of combinations starts to multiply rapidly.
What is needed is a way to describe these optional components using a partial Device Tree, and then to be able to build a complete tree by taking a base DT and adding a number of optional elements. You can do this, and these optional elements are called "overlays".

2.1: FRAGMENTS

A DT overlay comprises a number of fragments, each of which targets one node and its subnodes. Although the concept sounds simple enough, the syntax seems rather strange at first:
// Enable the i2s interface
/dts-v1/;
/plugin/;

/ {
    compatible = "brcm,bcm2708";

    fragment@0 {
        target = <&i2s>;
        __overlay__ {
            status = "okay";
        };
    };
};
The compatible string identifies this as being for BCM2708, which is the base architecture of the BCM2835 part. For the BCM2836 part you could use a compatible string of "brcm,bcm2709", but unless you are targeting features of the ARM CPUs, the two architectures should be equivalent, so sticking to "brcm,bcm2708" is reasonable. Then comes the first (and in this case only) fragment. Fragments are numbered sequentially from zero. Failure to adhere to this may cause some or all of your fragments to be missed.
Each fragment consists of two parts: a target property, identifying the node to apply the overlay to; and the __overlay__ itself, the body of which is added to the target node. The example above can be interpreted as if it were written like this:
/dts-v1/;

/ {
    compatible = "brcm,bcm2708";
};

&i2s {
    status = "okay";
};
The effect of merging that overlay with a standard Raspberry Pi base Device Tree (e.g. bcm2708-rpi-b-plus.dtb), provided the overlay is loaded afterwards, would be to enable the I2S interface by changing its status to okay. But if you try to compile this overlay using:
dtc -I dts -O dtb -o 2nd.dtbo 2nd-overlay.dts
you will get an error:
Label or path i2s not found
This shouldn't be too unexpected, since there is no reference to the base .dtb or .dts file to allow the compiler to find the i2s label.
Trying again, this time using the original example and adding the -@ option to allow unresolved references:
dtc -@ -I dts -O dtb -o 1st.dtbo 1st-overlay.dts
If dtc returns an error about the third line, it doesn't have the extensions required for overlay work. Run sudo apt-get install device-tree-compiler and try again - this time, compilation should complete successfully. Note that a suitable compiler is also available in the kernel tree as scripts/dtc/dtc, built when the dtbs make target is used:
make ARCH=arm dtbs
It is interesting to dump the contents of the DTB file to see what the compiler has generated:
$ fdtdump 1st.dtbo

/dts-v1/;
// magic:           0xd00dfeed
// totalsize:       0x106 (262)
// off_dt_struct:   0x38
// off_dt_strings:  0xe8
// off_mem_rsvmap:  0x28
// version:         17
// last_comp_version:    16
// boot_cpuid_phys: 0x0
// size_dt_strings: 0x1e
// size_dt_struct:  0xb0

/ {
    compatible = "brcm,bcm2708";
    fragment@0 {
        target = <0xdeadbeef>;
        __overlay__ {
            status = "okay";
        };
    };
    __fixups__ {
        i2s = "/fragment@0:target:0";
    };
};
After the verbose description of the file structure there is our fragment. But look carefully - where we wrote &i2s it now says 0xdeadbeef, a clue that something strange has happened. After the fragment there is a new node, __fixups__. This contains a list of properties mapping the names of unresolved symbols to lists of paths to cells within the fragments that need patching with the phandle of the target node, once that target has been located. In this case, the path is to the 0xdeadbeef value of target, but fragments can contain other unresolved references which would require additional fixes.
If you write more complicated fragments, the compiler may generate two more nodes: __local_fixups__ and __symbols__. The former is required if any node in the fragments has a phandle, because the program performing the merge will have to ensure that phandle numbers are sequential and unique. However, the latter is the key to how unresolved symbols are dealt with.
Back in section 1.3 it says that "the original labels do not appear in the compiled output", but this isn't true when using the -@ switch. Instead, every label results in a property in the __symbols__ node, mapping a label to a path, exactly like the aliases node. In fact, the mechanism is so similar that when resolving symbols, the Raspberry Pi loader will search the "aliases" node in the absence of a __symbols__ node. This is useful because by providing sufficient aliases, we can allow an older dtc to be used to build the base DTB files.
UPDATE: The Dynamic Device Tree support in the kernel requires a different format of "local fixups" in the overlay. To avoid problems with old and new styles of overlay coexisting, and to match other users of overlays, the old "name-overlay.dtb" naming scheme has been replaced with "name.dtbo" from 4.4 onwards. Overlays should be referred to by name alone, and the firmware or utility that loads them will append the appropriate suffix. For example:
dtoverlay=awesome-overlay      # This is wrong
dtoverlay=awesome              # This is correct

2.2: DEVICE TREE PARAMETERS

To avoid the need for lots of Device Tree overlays, and to reduce the need for users of peripherals to modify DTS files, the Raspberry Pi loader supports a new feature - Device Tree parameters. This permits small changes to the DT using named parameters, similar to the way kernel modules receive parameters from modprobe and the kernel command line. Parameters can be exposed by the base DTBs and by overlays, including HAT overlays.
Parameters are defined in the DTS by adding an __overrides__ node to the root. It contains properties whose names are the chosen parameter names, and whose values are a sequence comprising a phandle (reference to a label) for the target node, and a string indicating the target property; string, integer (cell) and boolean properties are supported.

2.2.1: STRING PARAMETERS

String parameters are declared like this:
name = <&label>,"property";
where label and property are replaced by suitable values. String parameters can cause their target properties to grow, shrink, or be created.
Note that properties called status are treated specially; non-zero/true/yes/on values are converted to the string "okay", while zero/false/no/off becomes "disabled".

2.2.2: INTEGER PARAMETERS

Integer parameters are declared like this:
name = <&label>,"property.offset"; // 8-bit
name = <&label>,"property;offset"; // 16-bit
name = <&label>,"property:offset"; // 32-bit
name = <&label>,"property#offset"; // 64-bit
where labelproperty and offset are replaced by suitable values; the offset is specified in bytes relative to the start of the property (in decimal by default), and the preceding separator dictates the size of the parameter. In a change from earlier implementations, integer parameters may refer to non-existent properties or to offsets beyond the end of an existing property.

2.2.3: BOOLEAN PARAMETERS

Device Tree encodes boolean values as zero-length properties; if present then the property is true, otherwise it is false. They are defined like this:
boolean_property; // Set 'boolean_property' to true
Note that a property is assigned the value false by not defining it. Boolean parameters are declared like this:
name = <&label>,"property?";
where label and property are replaced by suitable values. Boolean parameters can cause properties to be created or deleted.

2.2.4 OVERLAY/FRAGMENT PARAMETERS

The DT parameter mechanism as described has a number of limitations, including the inability to change the name of a node and to write arbitrary values to arbitrary properties when a parameter is used. One way to overcome some of these limitations is to conditionally include or exclude certain fragments.
A fragment can be excluded from the final merge process (disabled) by renaming the __overlay__ node to __dormant__. The parameter declaration syntax has been extended to allow the otherwise illegal zero target phandle to indicate that the following string contains operations at fragment or overlay scope. So far, four operations have been implemented:
+    // Enable fragment 
-    // Disable fragment 
=    // Enable fragment  if the assigned parameter value is true, otherwise disable it
!    // Enable fragment  if the assigned parameter value is false, otherwise disable it
Examples:
just_one    = <0>,"+1-2"; // Enable 1, disable 2
conditional = <0>,"=3!4"; // Enable 3, disable 4 if value is true,
                          // otherwise disable 3, enable 4.
The i2c-mux overlay uses this technique.

2.2.5 EXAMPLES

Here are some examples of different types of properties, with parameters to modify them:
/ {
    fragment@0 {
        target-path = "/";
        __overlay__ {

            test: test_node {
                string = "hello";
                status = "disabled";
                bytes = /bits/ 8 <0x67 0x89="">;
                u16s = /bits/ 16 <0xabcd 0xef01="">;
                u32s = /bits/ 32 <0xfedcba98 0x76543210="">;
                u64s = /bits/ 64 < 0xaaaaa5a55a5a5555 0x0000111122223333>;
                bool1; // Defaults to true
                       // bool2 defaults to false
            };
        };
    };

    fragment@1 {
        target-path = "/";
        __overlay__ {
            frag1;
        };
    };

    fragment@2 {
        target-path = "/";
        __dormant__ {
            frag2;
        };
    };

    __overrides__ {
        string =      <&test>,"string";
        enable =      <&test>,"status";
        byte_0 =      <&test>,"bytes.0";
        byte_1 =      <&test>,"bytes.1";
        u16_0 =       <&test>,"u16s;0";
        u16_1 =       <&test>,"u16s;2";
        u32_0 =       <&test>,"u32s:0";
        u32_1 =       <&test>,"u32s:4";
        u64_0 =       <&test>,"u64s#0";
        u64_1 =       <&test>,"u64s#8";
        bool1 =       <&test>,"bool1?";
        bool2 =       <&test>,"bool2?";
        only1 =       <0>,"+1-2";
        only2 =       <0>,"-1+2";
        toggle1 =     <0>,"=1";
        toggle2 =     <0>,"=2";
        not1 =        <0>,"!1";
        not2 =        <0>,"!2";
    };
};

2.2.6: PARAMETERS WITH MULTIPLE TARGETS

There are some situations where it is convenient to be able to set the same value in multiple locations within the Device Tree. Rather than the ungainly approach of creating multiple parameters, it is possible to add multiple targets to a single parameter by concatenating them, like this:
    __overrides__ {
        gpiopin = <&w1>,"gpios:4",
                  <&w1_pins>,"brcm,pins:0";
        ...
    };
(example taken from the w1-gpio overlay)
Note that it is even possible to target properties of different types with a single parameter. You could reasonably connect an "enable" parameter to a statusstring, cells containing zero or one, and a proper boolean property.

2.2.7: FURTHER OVERLAY EXAMPLES

There is a growing collection of overlay source files hosted in the Raspberry Pi/Linux GitHub repository here.

Part 3: Using Device Trees on Raspberry Pi

3.1: OVERLAYS AND CONFIG.TXT

On a Raspberry Pi it is the job of the loader (one of the start.elf images) to combine overlays with an appropriate base device tree, and then to pass a fully resolved Device Tree to the kernel. The base Device Trees are located alongside start.elf in the FAT partition (/boot from Linux), named bcm2708-rpi-b.dtbbcm2708-rpi-b-plus.dtbbcm2708-rpi-cm.dtb, and bcm2709-rpi-2-b.dtb. Note that Models A and A+ will use the "b" and "b-plus" variants, respectively. This selection is automatic, and allows the same SD card image to be used in a variety of devices.
Note that DT and ATAGs are mutually exclusive. As a result, passing a DT blob to a kernel that doesn't understand it causes a boot failure. To guard against this, the loader checks kernel images for DT-compatibility, which is marked by a trailer added by the mkknlimg utility; this can be found in the scripts directory of a recent kernel source tree. Any kernel without a trailer is assumed to be non-DT-capable.
A kernel built from the rpi-4.4.y tree (and later) will not function without a DTB, so from the 4.4 releases onwards, any kernel without a trailer is assumed to be DT-capable. You can override this by adding a trailer without the DTOK flag or by putting device_tree= in config.txt, but don't be surprised if it doesn't work. N.B. A corollary to this is that if the kernel has a trailer indicating DT capability then device_tree= will be ignored.
The loader now supports builds using bcm2835_defconfig, which selects the upstreamed BCM2835 support. This configuration will cause bcm2835-rpi-b.dtb and bcm2835-rpi-b-plus.dtb to be built. If these files are copied with the kernel, and if the kernel has been tagged by a recent mkknlimg, then the loader will attempt to load one of those DTBs by default.
In order to manage Device Tree and overlays, the loader supports a number of new config.txt directives:
dtoverlay=acme-board
dtparam=foo=bar,level=42
This will cause the loader to look for overlays/acme-board.dtbo in the firmware partition, which Raspbian mounts on /boot. It will then search for parameters foo and level, and assign the indicated values to them.
The loader will also search for an attached HAT with a programmed EEPROM, and load the supporting overlay from there; this happens without any user intervention.
There are several ways to tell that the kernel is using Device Tree:
  1. The "Machine model:" kernel message during bootup has a board-specific value such as "Raspberry Pi 2 Model B", rather than "BCM2709".
  2. Some time later, there may also be another kernel message saying "No ATAGs?" - this is expected.
  3. /proc/device-tree exists, and contains subdirectories and files that exactly mirror the nodes and properties of the DT.
With a Device Tree, the kernel will automatically search for and load modules that support the indicated enabled devices. As a result, by creating an appropriate DT overlay for a device you save users of the device from having to edit /etc/modules; all of the configuration goes in config.txt, and in the case of a HAT, even that step is unnecessary. Note, however, that layered modules such as i2c-dev still need to be loaded explicitly.
The flipside is that because platform devices don't get created unless requested by the DTB, it should no longer be necessary to blacklist modules that used to be loaded as a result of platform devices defined in the board support code. In fact, current Raspbian images ship without a blacklist file.

3.2: DT PARAMETERS

As described above, DT parameters are a convenient way to make small changes to a device's configuration. The current base DTBs support parameters for enabling and controlling the onboard audio, I2C, I2S and SPI interfaces without using dedicated overlays. In use, parameters look like this:
dtparam=audio=on,i2c_arm=on,i2c_arm_baudrate=400000,spi=on
Note that multiple assignments can be placed on the same line, but ensure you don't exceed the 80-character limit.
A future default config.txt may contain a section like this:
# Uncomment some or all of these to enable the optional hardware interfaces
#dtparam=i2c_arm=on
#dtparam=i2s=on
#dtparam=spi=on
If you have an overlay that defines some parameters, they can be specified either on subsequent lines like this:
dtoverlay=lirc-rpi
dtparam=gpio_out_pin=16
dtparam=gpio_in_pin=17
dtparam=gpio_in_pull=down
or appended to the overlay line like this:
dtoverlay=lirc-rpi:gpio_out_pin=16,gpio_in_pin=17,gpio_in_pull=down
Note here the use of a colon (:) to separate the overlay name from its parameters, which is a supported syntax variant.
Overlay parameters are only in scope until the next overlay is loaded. In the event of a parameter with the same name being exported by both the overlay and the base, the parameter in the overlay takes precedence; for clarity, it's recommended that you avoid doing this. To expose the parameter exported by the base DTB instead, end the current overlay scope using:
dtoverlay=

3.3: BOARD-SPECIFIC LABELS AND PARAMETERS

Raspberry Pi boards have two I2C interfaces. These are nominally split: one for the ARM, and one for VideoCore (the "GPU"). On almost all models, i2c1 belongs to the ARM and i2c0 to VC, where it is used to control the camera and read the HAT EEPROM. However, there are two early revisions of the Model B that have those roles reversed.
To make it possible to use one set of overlays and parameters with all Pis, the firmware creates some board-specific DT parameters. These are:
i2c/i2c_arm
i2c_vc
i2c_baudrate/i2c_arm_baudrate
i2c_vc_baudrate
These are aliases for i2c0i2c1i2c0_baudrate, and i2c1_baudrate. It is recommended that you only use i2c_vc and i2c_vc_baudrate if you really need to - for example, if you are programming a HAT EEPROM. Enabling i2c_vccan stop the Pi Camera being detected.
For people writing overlays, the same aliasing has been applied to the labels on the I2C DT nodes. Thus, you should write:
fragment@0 {
    target = <&i2c_arm>;
    __overlay__ {
        status = "okay";
    };
};
Any overlays using the numeric variants will be modified to use the new aliases.

3.4: HATS AND DEVICE TREE

A Raspberry Pi HAT is an add-on board for a "Plus"-shaped (A+, B+ or Pi 2 B) Raspberry Pi with an embedded EEPROM. The EEPROM includes any DT overlay required to enable the board, and this overlay can also expose parameters.
The HAT overlay is automatically loaded by the firmware after the base DTB, so its parameters are accessible until any other overlays are loaded, or until the overlay scope is ended using dtoverlay=. If for some reason you want to suppress the loading of the HAT overlay, put dtoverlay= before any other dtoverlay or dtparam directive.

3.5: DYNAMIC DEVICE TREE

As of Linux 4.4, the RPi kernels support the dynamic loading of overlays and parameters. Compatible kernels manage a stack of overlays that are applied on top of the base DTB. Changes are immediately reflected in /proc/device-tree and can cause modules to be loaded and platform devices to be created and destroyed.
The use of the word "stack" above is important - overlays can only be added and removed at the top of the stack; changing something further down the stack requires that anything on top of it must first be removed.
There are some new commands for managing overlays:

3.5.1 THE DTOVERLAY COMMAND

dtoverlay is a command line utility that loads and removes overlays while the system is running, as well as listing the available overlays and displaying their help information:
pi@raspberrypi ~ $ dtoverlay -h
Usage:
  dtoverlay  [=...]
                           Add an overlay (with parameters)
  dtoverlay -r [] Remove an overlay (by name, index or the last)
  dtoverlay -R [] Remove from an overlay (by name, index or all)
  dtoverlay -l             List active overlays/params
  dtoverlay -a             List all overlays (marking the active)
  dtoverlay -h             Show this usage message
  dtoverlay -h    Display help on an overlay
  dtoverlay -h  ..  Or its parameters
    where  is the name of an overlay or 'dtparam' for dtparams
Options applicable to most variants:
    -d     Specify an alternate location for the overlays
                (defaults to /boot/overlays or /flash/overlays)
    -n          Dry run - show what would be executed
    -v          Verbose operation
Unlike the config.txt equivalent, all parameters to an overlay must be included in the same command line - the dtparam command is only for parameters of the base DTB.
Two points to note:
  1. Command variants that change kernel state (adding and removing things) require root privilege, so you may need to prefix the command with sudo.
  2. Only overlays and parameters applied at run-time can be unloaded - an overlay or parameter applied by the firmware becomes "baked in" such that it won't be listed by dtoverlay and can't be removed.

3.5.2 THE DTPARAM COMMAND

dtparam creates an overlay that has the same effect as using a dtparam directive in config.txt. In usage it is largely equivalent to dtoverlay with an overlay name of -, but there are a few small differences:
  1. dtparam will list the help information for all known parameters of the base DTB. Help on the dtparam command is still available using dtparam -h.
  2. When indicating a parameter for removal, only index numbers can be used (not names).

3.5.3 GUIDELINES FOR WRITING RUNTIME-CAPABLE OVERLAYS

This area is poorly documented, but here are some accumulated tips:
  • The creation or deletion of a device object is triggered by a node being added or removed, or by the status of a node changing from disabled to enabled or vice versa. Beware - the absence of a "status" property means the node is enabled.
  • Don't create a node within a fragment that will overwrite an existing node in the base DTB - the kernel will rename the new node to make it unique. If you want to change the properties of an existing node, create a fragment that targets it.
  • ALSA doesn't prevent its codecs and other components from being unloaded while they are in use. Removing an overlay can cause a kernel exception if it deletes a codec that is still being used by a sound card. Experimentation found that devices are deleted in the reverse of fragment order in the overlay, so placing the node for the card after the nodes for the components allows an orderly shutdown.

3.5.4 CAVEATS

The loading of overlays at runtime is a recent addition to the kernel, and so far there is no accepted way to do this from userspace. By hiding the details of this mechanism behind commands the aim is to insulate users from changes in the event that a different kernel interface becomes standardised.
  • Some overlays work better at run-time than others. Parts of the Device Tree are only used at boot time - changing them using an overlay will not have any effect.
  • Applying or removing some overlays may cause unexpected behaviour, so it should be done with caution. This is one of the reasons it requires sudo.
  • Unloading the overlay for an ALSA card can stall if something is actively using ALSA - the LXPanel volume slider plugin demonstrates this effect. To enable overlays for sound cards to be removed, the lxpanelctl utility has been given two new options - alsastop and alsastart - and these are called from the auxiliary scripts dtoverlay-pre and dtoverlay-post before and after overlays are loaded or unloaded, respectively.
  • Removing an overlay will not cause a loaded module to be unloaded, but it may cause the reference count of some modules to drop to zero. Running rmmod -a twice will cause unused modules to be unloaded.
  • Overlays have to be removed in reverse order. The commands will allow you to remove an earlier one, but all the intermediate ones will be removed and re-applied, which may have unintended consequences.
  • Adding clocks under the /clocks node at run-time doesn't cause a new clock provider to be registered, so devm_clk_get will fail for a clock created in an overlay.

3.6: SUPPORTED OVERLAYS AND PARAMETERS

As it is too time-consuming to document the individual overlays here, please refer to the README file found alongside the overlay .dtbo files in /boot/overlays. It is kept up-to-date with additions and changes.

Part 4: Troubleshooting and pro tips

4.1: DEBUGGING

The loader will skip over missing overlays and bad parameters, but if there are serious errors, such as a missing or corrupt base DTB or a failed overlay merge, then the loader will fall back to a non-DT boot. If this happens, or if your settings don't behave as you expect, it is worth checking for warnings or errors from the loader:
sudo vcdbg log msg
Extra debugging can be enabled by adding dtdebug=1 to config.txt.
If the kernel fails to come up in DT mode, this is probably because the kernel image does not have a valid trailer. Use knlinfo to check for one, and the mkknlimg utility to add one. Both utilities are included in the scripts directory of current Raspberry Pi kernel source trees.
You can create a human-readable representation of the current state of DT like this:
dtc -I fs /proc/device-tree
This can be useful to see the effect of merging overlays onto the underlying tree.
If kernel modules don't load as expected, check that they aren't blacklisted in /etc/modprobe.d/raspi-blacklist.conf; blacklisting shouldn't be necessary when using Device Tree. If that shows nothing untoward, you can also check that the module is exporting the correct aliases by searching /lib/modules//modules.alias for the compatible value. Otherwise, your driver is probably missing either:
.of_match_table = xxx_of_match,
or:
MODULE_DEVICE_TABLE(of, xxx_of_match);
Failing that, depmod has failed or the updated modules haven't been installed on the target filesystem.

4.2: TESTING OVERLAYS USING DTMERGE AND DTDIFF

Alongside the dtoverlay and dtparam commands is a utility for applying an overlay to a DTB - dtmerge. To use it you first need to obtain your base DTB, which can be obtained in one of two ways:
a) generate it from the live DT state in /proc/device-tree:
dtc -I fs -O dtb -o base.dtb /proc/device-tree
This will include any overlays and parameters you have applied so far, either in config.txt or by loading them at runtime, which may or may not be what you want. Alternatively...
b) copy it from the source DTBs in /boot. This won't include overlays and parameters, but it also won't include any other modifications by the firmware. To allow testing of all overlays, the dtmerge utility will create some of the board-specific aliases ("i2c_arm", etc.), but this means that the result of a merge will include more differences from the original DTB than you might expect. The solution to this is to use dtmerge to make the copy:
dtmerge /boot/bcm2710-rpi-3-b.dtb base.dtb -
(the - indicates an absent overlay name).
You can now try applying an overlay or parameter:
dtmerge base.dtb merged.dtb - sd_overclock=62
dtdiff base.dtb merged.dtb
which will return:
--- /dev/fd/63  2016-05-16 14:48:26.396024813 +0100
+++ /dev/fd/62  2016-05-16 14:48:26.396024813 +0100
@@ -594,7 +594,7 @@
                };

                sdhost@7e202000 {
-                       brcm,overclock-50 = <0x0>;
+                       brcm,overclock-50 = <0x3e>;
                        brcm,pio-limit = <0x1>;
                        bus-width = <0x4>;
                        clocks = <0x8>;
You can also compare different overlays or parameters.
dtmerge base.dtb merged1.dtb /boot/overlays/spi1-1cs.dtbo
dtmerge base.dtb merged2.dtb /boot/overlays/spi1-2cs.dtbo
dtdiff merged1.dtb merged2.dtb
to get:
--- /dev/fd/63  2016-05-16 14:18:56.189634286 +0100
+++ /dev/fd/62  2016-05-16 14:18:56.189634286 +0100
@@ -453,7 +453,7 @@

                        spi1_cs_pins {
                                brcm,function = <0x1>;
-                               brcm,pins = <0x12>;
+                               brcm,pins = <0x12 0x11="">;
                                phandle = <0x3e>;
                        };

@@ -725,7 +725,7 @@
                        #size-cells = <0x0>;
                        clocks = <0x13 0x1="">;
                        compatible = "brcm,bcm2835-aux-spi";
-                       cs-gpios = <0xc 0x12="" 0x1="">;
+                       cs-gpios = <0xc 0x11="" 0x12="" 0x1="" 0xc="">;
                        interrupts = <0x1 0x1d="">;
                        linux,phandle = <0x30>;
                        phandle = <0x30>;
@@ -743,6 +743,16 @@
                                spi-max-frequency = <0x7a120>;
                                status = "okay";
                        };
+
+                       spidev@1 {
+                               #address-cells = <0x1>;
+                               #size-cells = <0x0>;
+                               compatible = "spidev";
+                               phandle = <0x41>;
+                               reg = <0x1>;
+                               spi-max-frequency = <0x7a120>;
+                               status = "okay";
+                       };
                };

                spi@7e2150C0 {

4.3: FORCING A SPECIFIC DEVICE TREE

If you have very specific needs that aren't supported by the default DTBs (in particular, people experimenting with the pure-DT approach used by the ARCH_BCM2835 project), or if you just want to experiment with writing your own DTs, you can tell the loader to load an alternate DTB file like this:
device_tree=my-pi.dtb

4.4: DISABLING DEVICE TREE USAGE

Since the switch to the 4.4 kernel and the use of more upstream drivers, Device Tree usage is required in Pi kernels. The method of disabling DT usage is to add:
device_tree=
to config.txt. However, if the kernel has a mkknlimg trailer indicating DT capability then this directive will be ignored.

4.5: SHORTCUTS AND SYNTAX VARIANTS

The loader understands a few shortcuts:
dtparam=i2c_arm=on
dtparam=i2s=on
can be shortened to:
dtparam=i2c,i2s
(i2c is an alias of i2c_arm, and the =on is assumed). It also still accepts the long-form versions: device_tree_overlay and device_tree_param.
The loader used to accept the use of whitespace and colons as separators, but support for these has been discontinued for simplicity and so they can be used within parameter values without quotes.

4.6: OTHER DT COMMANDS AVAILABLE IN CONFIG.TXT

device_tree_address This is used to override the address where the firmware loads the device tree (not dt-blob). By default the firmware will choose a suitable place.
device_tree_end This sets an (exclusive) limit to the loaded device tree. By default the device tree can grow to the end of usable memory, which is almost certainly what is required.
dtdebug If non-zero, turn on some extra logging for the firmware's device tree processing.
enable_uart Enable the primary/console UART (ttyS0 on a Pi 3, ttyAMA0 otherwise - unless swapped with an overlay such as pi3-miniuart-bt). If the primary UART is ttyAMA0 then enable_uart defaults to 1 (enabled), otherwise it defaults to 0 (disabled). This is because it is necessary to stop the core frequency from changing which would make ttyS0 unusable, so enable_uart=1 implies core_freq=250 (unless force_turbo=1). In some cases this is a performance hit, so it is off by default. More details on UARTs can be found here
overlay_prefix Specifies a subdirectory/prefix from which to load overlays - defaults to "overlays/". Note the trailing "/". If desired you can add something after the final "/" to add a prefix to each file, although this is not likely to be needed.
Further ports can be controlled by the DT, for more details see section 3.