diff options
Diffstat (limited to 'docs/NOTES')
| -rw-r--r-- | docs/NOTES | 311 |
1 files changed, 311 insertions, 0 deletions
diff --git a/docs/NOTES b/docs/NOTES new file mode 100644 index 0000000..d87bbfc --- /dev/null +++ b/docs/NOTES @@ -0,0 +1,311 @@ +Memory Addressing +================= + There is 3 types of addresses: virtual, physical, and bus. For DMA a bus + address is used. However, on x86 physical and bus addresses are the same (on + other architectures it is not guaranteed). Anyway, this assumption is still + used by xdma driver, it uses phiscal address for DMA access. I have ported + in the same way. Now, we need to provide additionaly bus-addresses in kmem + abstraction and use it in NWL DMA implementation. + +DMA Access Synchronization +========================== + - At driver level, few types of buffers are supported: + * SIMPLE - non-reusable buffers, the use infomation can be used for cleanup + after crashed applications. + * EXCLUSIVE - reusable buffers which can be mmaped by a single appliction + only. There is two modes of these buffers: + + Buffers in a STANDARD mode are created for a single DMA operation and + if such buffer is detected while trying to reuse, the last operation + has failed and reset is needed. + + Buffers in a PERSISTENT mode are preserved between invocations of + control application and cleaned up only after the PERSISTENT flag is + removed + * SHARED - reusable buffers shared by multiple processes. Not really + needed at the moment. + + KMEM_FLAG_HW - indicates that buffer can be used by hardware, acually this + means that DMA will be enabled afterwards. The driver is not able to check + if it really was enable and therefore will block any attempt to release + buffer until KMEM_HW_FLAG is passed to kmem_free routine as well. The later + should only called with KMEM_HW_FLAG after the DMA engine is stopped. Then, + the driver can be realesd by kmem_free if ref count reaches 0. + + KMEM_FLAG_EXCLUSIVE - prevents multiple processes mmaping the buffer + simultaneously. This is used to prevent multiple processes use the same + DMA engine at the same time. When passed to kmem_free, allows to clean + buffers with lost clients even for shared buffers. + + KMEM_FLAG_REUSE - requires reuse of existing buffer. If reusable buffer is + found (non-reusable buffers, i.e. allocated without KMEM_FLAG_REUSE are + ignored), it is returned instead of allocation. Three types of usage + counters are used. At moment of allocation, the HW reference is set if + neccessary. The usage counter is increased by kmem_alloc function and + decreased by kmem_free. Finally, the reference is obtained at returned + during mmap/munmap. So, on kmem_free, we do not clean + a) buffers with reference count above zero or hardware reference set. + REUSE flag should be supplied, overwise the error is returned + b) PERSISTENT buffer. REUSE flash should be supplied, overwise the + error is returned + c) non-exclusive buffers with usage counter above zero (For exclusive + buffer the value of usage counter above zero just means that application + have failed without cleaning buffers first. There is no easy way to + detect that for shared buffers, so it is left as manual operation in + this case) + d) any buffer if KMEM_FLAG_REUSE was provided to function + During module unload, only buffers with references can prevent cleanup. In + this case the only possiblity to free the driver is to call kmem_free + passing FORCE flags. + + KMEM_FLAG_PERSISTENT - if passed to allocation routine, changes mode of + buffer to PERSISTENT, if passed to free routine, vice-versa changes mode + of buffer to NORMAL. Basically, if we call 'pci --dma-start' this flag + should be passed to alloc and if we call 'pci --dma-stop' it should be + passed to free. In other case, the flag should not be present. + + If application crashed, the munmap while be still called cleaning software + references. However, the hardware reference will stay since it is not clear + if hardware channel was closed or not. To lift hardware reference, the + application can be re-executed (or dma_stop called, for instance). + * If there is no hardware reference, the buffers will be reused by next + call to application and for EXCLUSIVE buffer cleaned at the end. For SHARED + buffers they will be cleaned during module cleanup only (no active + references). + * The buffer will be reused by next call which can result in wrong behaviour + if buffer left in incoherent stage. This should be handled on upper level. + + - At pcilib/kmem level synchronization of multiple buffers is performed + * The HW reference and following modes should be consistent between member + parts: REUSABLE, PERSISTENT, EXCLUSIVE (only HW reference and PERSISTENT + mode should be checked, others are handled on dirver level) + * It is fine if only part of buffers are reused and others are newly + allocated. However, on higher level this can be checked and resulting + in failure. + + Treatment of inconsistencies: + * Buffers are in PRESISTENT mode, but newly allocated, OK + * Buffers are reused, but are not in PERSISTENT mode (for EXCLUSIVE buffers + this means that application has crashed during the last execution), OK + * Some of buffers are reused (not just REUSABLE, but actually reused), + others - not, OK until + a) either PERSISTENT flag is set or reused buffers are non-PERSISTENT + b) either HW flag is set or reused buffers does not hold HW reference + * PERSISTENT mode inconsistency, FAIL (even if we are going to set + PERSISTENT mode anyway) + * HW reference inconsistency, FAIL (even if we are going to set + HW flag anyway) + + On allocation error at some of the buffer, call clean routine and + * Preserve PERSISTENT mode and HW reference if buffers held them before + unsuccessful kmem initialization. Until the last failed block, the blocks + of kmem should be consistent. The HW/PERSISTENT flags should be removed + if all reused blocks were in HW/PERSISTENT mode. The last block needs + special treatment. The flags may be removed for the block if it was + HW/PERSISTENT state (and others not). + * Remove REUSE flag, we want to clean if allowed by current buffer status + * EXCLUSIVE flag is not important for kmem_free routine. + + - At DMA level + There is 4 components of DMA access: + * DMA engine enabled/disabled + * DMA engine IRQs enabled/disabled - always enabled at startup + * Memory buffers + * Ring start/stop pointers + + To prevent multiple processes accessing DMA engine in parallel, the first + action is buffer initialization which will fail if buffers already used + * Always with REUSE, EXCLUSIVE, and HW flags + * Optionally with PERSISTENT flag (if DMA_PERSISTENT flag is set) + If another DMA app is running, the buffer allocation will fail (no dma_stop + is executed in this case) + + Depending on PRESERVE flag, kmem_free will be called with REUSE flag + keeping buffer in memory (this is redundant since HW flag is enough) or HW + flag indicating that DMA engine is stopped and buffer could be cleaned. + PERSISTENT flag is defined by DMA_PERSISTENT flag passed to stop routine. + + PRESERVE flag is enforced if DMA_PERSISTENT is not passed to dma_stop + routine and either it: + a) Explicitely set by DMA_PERMANENT flag passed to dma_start + function + b) Implicitely set if DMA engine is already enabled during dma_start, + all buffers are reused, and are in persistent mode. + If PRESERVE flag is on, the engine will not be stopped at the end of + execution (and buffers will stay because of HW flag). + + If buffers are reused and are already in PERSISTENT mode, DMA engine was on + before dma_start (PRESERVE flag is ignored, because it can be enforced), + ring pointers are calculated from LAST_BD and states of ring elements. + If previous application crashed (i.e. buffers may be corrupted). Two + cases are possible: + * If during the call buffers were in non-PERSISTENT mode, it can be + easily detected - buffers are reused, but are not in PERSISTENT mode + (or at least was not before we set them to). In this case we just + reinitialize all buffers. + * If during the call buffers were in PERSISTENT mode, it is up to + user to check their consistency and restart DMA engine.] + + IRQs are enabled and disabled at each call + +DMA Reads +========= +standard: default reading mode, reads a single full packet +multipacket: reads all available packets +waiting multipacket: reads all available packets, after finishing the + last one waiting if new data arrives +exact read: read exactly specified number of bytes (should be + only supported if it is multiple of packets, otherwise + error should be returned) +ignore packets: autoterminate each buffer, depends on engine + configuration + + To handle differnt cases, the value returned by callback function instructs +the DMA library how long to wait for the next data to appear before timing +out. The following variants are possible: +terminate: just bail out +check: no timeout, just check if there is data, otherwise + terminate +timeout: standard DMA timeout, normaly used while receiving + fragments of packet: in this case it is expected + that device has already prepared data and only + the performance of DMA engine limits transfer speed +wait: wait until the data is prepared by the device, this + timeout is specified as argument to the dma_stream + function (standard DMA timeout is used by default) + + first | new_pkt | bufer + -------------------------- +standard wait | term | timeout +multiple packets wait | check | timeout - DMA_READ_FLAG_MULTIPACKET +waiting multipacket wait | wait | timeout - DMA_READ_FLAG_WAIT +exact wait | wait/term | timeout - limited by size parameter +ignore packets wait | wait/check| wait/check - just autoterminated + +Shall we do a special handling in case of overflow? + + +Buffering +========= + The DMA addresses are limited to 32 bits (~4GB for everything). This means we + can't really use DMA pages are sole buffers. Therefore, a second thread, with + a realtime scheduling policy if possible, will be spawned and will copy the + data from the DMA pages into the allocated buffers. On expiration of duration + or number of events set by autostop call, this thread will be stopped but + processing in streaming mode will continue until all copyied data is passed + to the callbacks. + + To avoid stalls, the IPECamera requires data to be read continuously read out. + For this reason, there is no locks in the readout thread. It will simplify + overwrite the old frames if data is not copied out timely. To handle this case + after getting the data and processing it, the calling application should use + return_data function and check return code. This function may return error + indicating that the data was overwritten meanwhile. Hence, the data is + corrupted and shoud be droped by the application. The copy_data function + performs this check and user application can be sure it get coherent data + in this case. + + There is a way to avoid this problem. For raw data, the rawdata callback + can be requested. This callback blocks execution of readout thread and + data may be treated safely by calling application. However, this may + cause problems to electronics. Therefore, only memcpy should be performed + on the data normally. + + The reconstructed data, however, may be safely accessed. As described above, + the raw data will be continuously overwritten by the reader thread. However, + reconstructed data, upon the get_data call, will be protected by the mutex. + + +Register Access Synchronization +=============================== + We need to serialize access to the registers by the different running + applications and handle case when registers are accessed indirectly by + writting PCI BARs (DMA implementations, for instance). + + - Module-assisted locking: + * During initialization the locking context is created (which is basicaly + a kmem_handle of type LOCK_PAGE. + * This locking context is passed to the kernel module along with lock type + (LOCK_BANK) and lock item (BANK ADDRESS). If lock context is already owns + lock on the specified bank, just reference number is increased, otherwise + we are trying to obtain new lock. + * Kernel module just iterates over all registered lock pages and checks if + any holds the specified lock. if not, the lock is obtained and registered + in the our lock page. + * This allows to share access between multiple threads of single application + (by using the same lock page) or protect (by using own lock pages by each of + the threads) + * Either on application cleanup or if application crashed, the memory mapping + of lock page is removed and, hence, locks are freed. + + - Multiple-ways of accessing registers + Because of reference counting, we can successfully obtain locks multiple + times if necessary. The following locks are protecting register access: + a) Global register_read/write lock bank before executing implementation + b) DMA bank is locked by global DMA functions. So we can access the + registers using plain PCI bar read/write. + c) Sequence of register operations can be protected with pcilib_lock_bank + function + Reading raw register space or PCI bank is not locked. + * Ok. We can detect banks which will be affected by PCI read/write and + lock them. But shall we do it? + +Register/DMA Configuration +========================== + - XML description of registers + - Formal XML-based (or non XML-based) language for DMA implementation. + a) Writting/Reading register values + b) Wait until <register1>=<value> on <register2>=<value> report error + c) ... ? + +IRQ Handling +============ + IRQ types: DMA IRQ, Event IRQ, other types + IRQ hardware source: To allow purely user-space implementation, as general + rule, only a single (standard) source should be used. + IRQ source: The dma/event engines, however, may detail this hardware source + and produce real IRQ source basing on the values of registers. For example, + for DMA IRQs the source may present engine number and for Event IRQs the + source may present event type. + + Only types can be enabled or disabled. The sources are enabled/disabled + by enabling/disabling correspondent DMA engines or Event types. The expected + workflow is following: + * We enabling IRQs in user-space (normally setting some registers). Normally, + just an Event IRQs, the DMA if necessary will be managed by DMA engine itself. + * We waiting for standard IRQ from hardware (driver) + * In the user space, we are checking registers to find out the real source + of IRQ (driver reports us just hardware source), generating appropriate + events, and acknowledge IRQ. This is dependent on implementation and should + be managed inside event API. + + I.e. the driver implements just two methods pcilib_wait_irq(hw_source), + pcilib_clear_irq(hw_source). Only a few hardware IRQ sources are defined. + In most cirstumances, the IRQ_SOURCE_DEFAULT is used. + + The DMA engine may provide 3 additional methods, to enable, disable, + and acknowledge IRQ. + + ... To be decided in details upon the need... + +Updating Firmware +================= + - JTag should be connected to USB connector on the board (next to Ethernet) + - The computer should be tourned off and on before programming + - The environment variable should be loaded + . /home/uros/.bashrc + - The application is called 'impact' + No project is needed, cancel initial proposals (No/Cancel) + Double-click on "Boundary Scan" + Right click in the right window and select "Init Chain" + We don't want to select bit file now (Yes and, then, click Cancel) + Right click on second (right) item and choose "Assign new CF file" + Select a bit file. Answer No, we don't want to attach SPI to SPI Prom + Select xv6vlx240t and program it + - Shutdown and start computer + + Firmware are in + v.2: /home/uros/Repo/UFO2_last_good_version_UFO2.bit + v.3: /home/uros/Repo/UFO3 + Step5 - best working revision + Step6 - last revision + +
\ No newline at end of file |