Skip to content

Formalize the ProgressReporting Type #4335

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jun 30, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion ghcide/src/Development/IDE/Core/Compile.hs
Original file line number Diff line number Diff line change
Expand Up @@ -69,9 +69,10 @@
import Data.Time (UTCTime (..))
import Data.Tuple.Extra (dupe)
import Debug.Trace
import Development.IDE.Core.FileStore (resetInterfaceStore)

Check warning on line 72 in ghcide/src/Development/IDE/Core/Compile.hs

View workflow job for this annotation

GitHub Actions / Hlint check run

Warning in module Development.IDE.Core.Compile: Use fewer imports ▫︎ Found: "import Development.IDE.Core.FileStore ( resetInterfaceStore )\nimport Development.IDE.Core.FileStore ( shareFilePath )\n" ▫︎ Perhaps: "import Development.IDE.Core.FileStore\n ( resetInterfaceStore, shareFilePath )\n"
import Development.IDE.Core.Preprocessor
import Development.IDE.Core.ProgressReporting (ProgressReporting (..))
import Development.IDE.Core.ProgressReporting (ProgressReporting (..),
progressReportingNoTrace, progressUpdate)
import Development.IDE.Core.RuleTypes
import Development.IDE.Core.Shake
import Development.IDE.Core.Tracing (withTrace)
Expand Down Expand Up @@ -956,7 +957,7 @@


convImport (L _ i) = (
(ideclPkgQual i)

Check warning on line 960 in ghcide/src/Development/IDE/Core/Compile.hs

View workflow job for this annotation

GitHub Actions / Hlint check run

Suggestion in getModSummaryFromImports in module Development.IDE.Core.Compile: Redundant bracket ▫︎ Found: "((ideclPkgQual i), reLoc $ ideclName i)" ▫︎ Perhaps: "(ideclPkgQual i, reLoc $ ideclName i)"
, reLoc $ ideclName i)

msrImports = implicit_imports ++ imps
Expand Down
114 changes: 64 additions & 50 deletions ghcide/src/Development/IDE/Core/ProgressReporting.hs
Original file line number Diff line number Diff line change
@@ -1,15 +1,21 @@
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TypeFamilies #-}

module Development.IDE.Core.ProgressReporting
( ProgressEvent (..),
ProgressReporting (..),
ProgressReportingNoTrace,
noProgressReporting,
progressReporting,
progressReportingOutsideState,
progressReportingNoTrace,
-- utilities, reexported for use in Core.Shake
mRunLspT,
mRunLspTCallback,
-- for tests
recordProgress,
InProgressState (..),
progressStop,
progressUpdate
)
where

Expand Down Expand Up @@ -42,38 +48,60 @@ data ProgressEvent
| ProgressCompleted
| ProgressStarted

data ProgressReporting m = ProgressReporting
{ progressUpdate :: ProgressEvent -> m (),
inProgress :: forall a. NormalizedFilePath -> m a -> m a,
-- ^ see Note [ProgressReporting API and InProgressState]
progressStop :: IO ()
data ProgressReportingNoTrace m = ProgressReportingNoTrace
{ progressUpdateI :: ProgressEvent -> m (),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might not need to add name suffixes if you use DuplicateRecordFields?

Copy link
Collaborator Author

@soulomoon soulomoon Jun 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nop, we have class function and data field collision. DuplicateRecordFields is not helpful here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But maybe we can make it prefix with '_' and not export it.

progressStopI :: IO ()
-- ^ we are using IO here because creating and stopping the `ProgressReporting`
-- is different from how we use it.
}

data ProgressReporting m = ProgressReporting
{
inProgress :: forall a. NormalizedFilePath -> m a -> m a,
-- ^ see Note [ProgressReporting API and InProgressState]
progressReportingInner :: ProgressReportingNoTrace m
}

class ProgressReportingClass a where
type M a :: * -> *
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I asked this before, but can everything just be in IO? The only place we use another monad is in inProgress, and I think we could also just provide that with IO and get people to use MonadUnliftIO to use it?

Copy link
Collaborator Author

@soulomoon soulomoon Jun 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can turn them into IOs and save the m for type cleaness, but might introuce a bit runtime overhead?

progressUpdate :: a -> ProgressEvent -> M a ()
progressStop :: a -> IO ()

instance ProgressReportingClass (ProgressReportingNoTrace m) where
type M (ProgressReportingNoTrace m) = m
progressUpdate = progressUpdateI
progressStop = progressStopI

instance ProgressReportingClass (ProgressReporting m) where
type M (ProgressReporting m) = m
progressUpdate :: ProgressReporting m -> ProgressEvent -> M (ProgressReporting m) ()
progressUpdate = progressUpdateI . progressReportingInner
progressStop = progressStopI . progressReportingInner

{- Note [ProgressReporting API and InProgressState]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The progress of tasks can be tracked in two ways:

1. `InProgressState`: This is an internal state that actively tracks the progress.
1. `ProgressReporting`: we have an internal state that actively tracks the progress.
Changes to the progress are made directly to this state.

2. `InProgressStateOutSide`: This is an external state that tracks the progress.
2. `ProgressReportingNoTrace`: there is an external state that tracks the progress.
The external state is converted into an STM Int for the purpose of reporting progress.

The `inProgress` function is only useful when we are using `InProgressState`.

An alternative design could involve using GADTs to eliminate this discrepancy between
`InProgressState` and `InProgressStateOutSide`.
The `inProgress` function is only useful when we are using `ProgressReporting`.
-}

noProgressReportingNoTrace :: (MonadUnliftIO m) => (ProgressReportingNoTrace m)
noProgressReportingNoTrace = ProgressReportingNoTrace
{ progressUpdateI = const $ pure (),
progressStopI = pure ()
}
noProgressReporting :: (MonadUnliftIO m) => IO (ProgressReporting m)
noProgressReporting =
return $
ProgressReporting
{ progressUpdate = const $ pure (),
inProgress = const id,
progressStop = pure ()
{ inProgress = const id,
progressReportingInner = noProgressReportingNoTrace
}

-- | State used in 'delayedProgressReporting'
Expand Down Expand Up @@ -106,19 +134,11 @@ data InProgressState
doneVar :: TVar Int,
currentVar :: STM.Map NormalizedFilePath Int
}
| InProgressStateOutSide
-- we transform the outside state into STM Int for progress reporting purposes
{ -- | Number of files to do
todo :: STM Int,
-- | Number of files done
done :: STM Int
}

newInProgress :: IO InProgressState
newInProgress = InProgressState <$> newTVarIO 0 <*> newTVarIO 0 <*> STM.newIO

recordProgress :: InProgressState -> NormalizedFilePath -> (Int -> Int) -> IO ()
recordProgress InProgressStateOutSide {} _ _ = return ()
recordProgress InProgressState {..} file shift = do
(prev, new) <- atomicallyNamed "recordProgress" $ STM.focus alterPrevAndNew file currentVar
atomicallyNamed "recordProgress2" $ do
Expand All @@ -138,50 +158,44 @@ recordProgress InProgressState {..} file shift = do
alter x = let x' = maybe (shift 0) shift x in Just x'


-- | `progressReporting` initiates a new progress reporting session.
-- It necessitates the active tracking of progress using the `inProgress` function.
-- Refer to Note [ProgressReporting API and InProgressState] for more details.
progressReporting ::
(MonadUnliftIO m, MonadIO m) =>
Maybe (LSP.LanguageContextEnv c) ->
T.Text ->
ProgressReportingStyle ->
IO (ProgressReporting m)
progressReporting = progressReporting' newInProgress

-- | `progressReportingOutsideState` initiates a new progress reporting session.
-- | `progressReportingNoTrace` initiates a new progress reporting session.
-- It functions similarly to `progressReporting`, but it utilizes an external state for progress tracking.
-- Refer to Note [ProgressReporting API and InProgressState] for more details.
progressReportingOutsideState ::
progressReportingNoTrace ::
(MonadUnliftIO m, MonadIO m) =>
STM Int ->
STM Int ->
Maybe (LSP.LanguageContextEnv c) ->
T.Text ->
ProgressReportingStyle ->
IO (ProgressReporting m)
progressReportingOutsideState todo done = progressReporting' (pure $ InProgressStateOutSide todo done)
IO (ProgressReportingNoTrace m)
progressReportingNoTrace _ _ Nothing _title _optProgressStyle = return noProgressReportingNoTrace
progressReportingNoTrace todo done (Just lspEnv) title optProgressStyle = do
progressState <- newVar NotStarted
let progressUpdateI event = liftIO $ updateStateVar $ Event event
progressStopI = updateStateVar StopProgress
updateStateVar = modifyVar_ progressState . updateState (progressCounter lspEnv title optProgressStyle todo done)
return ProgressReportingNoTrace {..}

progressReporting' ::
-- | `progressReporting` initiates a new progress reporting session.
-- It necessitates the active tracking of progress using the `inProgress` function.
-- Refer to Note [ProgressReporting API and InProgressState] for more details.
progressReporting ::
forall c m.
(MonadUnliftIO m, MonadIO m) =>
IO InProgressState ->
Maybe (LSP.LanguageContextEnv c) ->
T.Text ->
ProgressReportingStyle ->
IO (ProgressReporting m)
progressReporting' _newState Nothing _title _optProgressStyle = noProgressReporting
progressReporting' newState (Just lspEnv) title optProgressStyle = do
inProgressState <- newState
progressState <- newVar NotStarted
let progressUpdate event = liftIO $ updateStateVar $ Event event
progressStop = updateStateVar StopProgress
updateStateVar = modifyVar_ progressState . updateState (lspShakeProgressNew inProgressState)
progressReporting Nothing _title _optProgressStyle = noProgressReporting
progressReporting (Just lspEnv) title optProgressStyle = do
inProgressState <- newInProgress
progressReportingInner <- progressReportingNoTrace (readTVar $ todoVar inProgressState)
(readTVar $ doneVar inProgressState) (Just lspEnv) title optProgressStyle
let inProgress :: forall a. NormalizedFilePath -> m a -> m a
inProgress = updateStateForFile inProgressState
return ProgressReporting {..}
where
lspShakeProgressNew :: InProgressState -> IO ()
lspShakeProgressNew InProgressStateOutSide {..} = progressCounter lspEnv title optProgressStyle todo done
lspShakeProgressNew InProgressState {..} = progressCounter lspEnv title optProgressStyle (readTVar todoVar) (readTVar doneVar)
updateStateForFile inProgress file = UnliftIO.bracket (liftIO $ f succ) (const $ liftIO $ f pred) . const
where
-- This functions are deliberately eta-expanded to avoid space leaks.
Expand Down
4 changes: 2 additions & 2 deletions ghcide/src/Development/IDE/Core/Shake.hs
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,7 @@ data HieDbWriter
{ indexQueue :: IndexQueue
, indexPending :: TVar (HMap.HashMap NormalizedFilePath Fingerprint) -- ^ Avoid unnecessary/out of date indexing
, indexCompleted :: TVar Int -- ^ to report progress
, indexProgressReporting :: ProgressReporting IO
, indexProgressReporting :: ProgressReportingNoTrace IO
}

-- | Actions to queue up on the index worker thread
Expand Down Expand Up @@ -676,7 +676,7 @@ shakeOpen recorder lspEnv defaultConfig idePlugins debouncer
indexPending <- newTVarIO HMap.empty
indexCompleted <- newTVarIO 0
semanticTokensId <- newTVarIO 0
indexProgressReporting <- progressReportingOutsideState
indexProgressReporting <- progressReportingNoTrace
(liftM2 (+) (length <$> readTVar indexPending) (readTVar indexCompleted))
(readTVar indexCompleted)
lspEnv "Indexing" optProgressStyle
Expand Down
Loading