Skip to content

Commit 8e04356

Browse files
committed
Shuffle CNNDM dataloader
1 parent d6ce2f9 commit 8e04356

File tree

1 file changed

+7
-9
lines changed

1 file changed

+7
-9
lines changed

beginner_source/t5_tutorial.py

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,7 @@ def apply_prefix(task, x):
133133
cnndm_datapipe = cnndm_datapipe.map(partial(apply_prefix, task))
134134
cnndm_datapipe = cnndm_datapipe.batch(cnndm_batch_size)
135135
cnndm_datapipe = cnndm_datapipe.rows2columnar(["article", "abstract"])
136-
cnndm_dataloader = DataLoader(cnndm_datapipe, batch_size=None)
136+
cnndm_dataloader = DataLoader(cnndm_datapipe, shuffle=True, batch_size=None)
137137

138138
#######################################################################
139139
# Alternately, we can also use batched API, for example, apply the prefix on the whole batch:
@@ -223,21 +223,19 @@ def process_labels(labels, x):
223223

224224

225225
#######################################################################
226-
# Summarization Output
226+
# Summarization Output (Might vary since we shuffle the dataloader)
227227
# --------------------
228228
#
229229
# .. code-block::
230230
#
231231
# Example 1:
232232
#
233-
# prediction: the Palestinians become the 123rd member of the international criminal
234-
# court . the accession was marked by a ceremony at the Hague, where the court is based .
235-
# the ICC opened a preliminary examination into the situation in the occupied
236-
# Palestinian territory .
233+
# prediction: the 24-year-old has been tattooed for over a decade . he has landed in australia
234+
# to start work on a new campaign . he says he is 'taking it in your stride' to be honest .
237235
#
238-
# target: Membership gives the ICC jurisdiction over alleged crimes committed in
239-
# Palestinian territories since last June . Israel and the United States opposed the
240-
# move, which could open the door to war crimes investigations against Israelis .
236+
# target: London-based model Stephen James Hendry famed for his full body tattoo . The supermodel
237+
# is in Sydney for a new modelling campaign . Australian fans understood to have already located
238+
# him at his hotel . The 24-year-old heartthrob is recently single .
241239
#
242240
#
243241
# Example 2:

0 commit comments

Comments
 (0)