Skip to content

Commit 0f9a944

Browse files
authored
Fixed the FOBS doc to add support for DatumManager (#2408)
1 parent 0966722 commit 0f9a944

File tree

1 file changed

+7
-2
lines changed

1 file changed

+7
-2
lines changed

nvflare/fuel/utils/fobs/README.rst

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,11 @@ A decomposer can either serialize the class into bytes or decompose it into obje
117117
serializable types. In most cases, it only involves saving members as a list and reconstructing
118118
the object from the list.
119119

120+
MessagePack can't handle items larger than 4GB in dict. To work around this issue, FOBS can externalize
121+
the large item and just stores a reference in the buffer. :code:`DatumManager` is used to handle the
122+
externalized data. For most objects which don't deal with dict items larger than 4GB, the DatumManager
123+
is not needed.
124+
120125
Here is an example of a simple decomposer. Even though :code:`datetime` is not supported
121126
by MessagePack, a decomposer is included in `fobs` module so no need to further decompose it.
122127

@@ -138,10 +143,10 @@ by MessagePack, a decomposer is included in `fobs` module so no need to further
138143
def supported_type(self) -> Type[Any]:
139144
return Simple
140145

141-
def decompose(self, obj) -> Any:
146+
def decompose(self, obj, manager) -> Any:
142147
return [obj.num, obj.name, obj.timestamp]
143148

144-
def recompose(self, data: Any) -> Simple:
149+
def recompose(self, data: Any, manager) -> Simple:
145150
return Simple(data[0], data[1], data[2])
146151

147152

0 commit comments

Comments
 (0)