Skip to content

Fix create issue script #2876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Nov 21, 2024
Merged
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 116 additions & 41 deletions scripts/create_issue.py
Original file line number Diff line number Diff line change
@@ -1,63 +1,138 @@
# Use together with `pageviews.py`
# python scripts/pageviews.py | head -n 150 | grep -v whats | cut -d ' ' -f 2 | sed 's/\.html/\.po/g' | xargs -I '{}' python scripts/create_issue.py '{}'
"""
Run this script with one variable:
- PO filename to create an issue for that file
- or '--all' to create the issues for all untranslated files that doesn't have an open issue already
- or '--one' to create the next one issue
"""

import os
import sys
from glob import glob
from pathlib import Path

from github import Github
from potodo.potodo import PoFileStats

if len(sys.argv) != 2:
print('Specify PO filename')
sys.exit(1)

pofilename = sys.argv[1]
pofile = PoFileStats(Path(pofilename))

g = Github(os.environ.get('GITHUB_TOKEN'))

repo = g.get_repo('python/python-docs-es')

PYTHON_VERSION = "3.13"
ISSUE_LABELS = [PYTHON_VERSION, "good first issue"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

El "good first issue" yo lo aplicaría sólo para issues donde jay que traducir menos de N entradas (5, 10?) en vez de aplicarlo de forma indiscriminada a todos

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Me gustó la idea, done acá: 1915095

ISSUE_TITLE = 'Translate `{pofilename}`'
ISSUE_BODY = '''This needs to reach 100% translated.

issues = repo.get_issues(state='all')
for issue in issues:
if pofilename in issue.title:
The rendered version of this file will be available at https://docs.python.org/es/{python_version}/{urlfile} once translated.
Meanwhile, the English version is shown.

print(f'Skipping {pofilename}. There is a similar issue already created at {issue.html_url}')
sys.exit(1)
Current stats for `{pofilename}`:

msg = f'There is a similar issue already created at {issue.html_url}.\nDo you want to create it anyways? [y/N] '
answer = input(msg)
if answer != 'y':
sys.exit(1)
- Fuzzy: {pofile_fuzzy}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

La forma en que estas estadísticas se muestran siempre me ha parecido un poco confusa, porque hay que hacer un poco de matemáticas para entender cuántas entradas de verdad necesitan trabajo. Parte de esa confusión (creo) también es que las entradas fuzzy creo que cuentan para el pofile.percent_translated, pero puedo estar equivocando (o quizás ha cambiado eso en potodo).

Yo mostraría la información más o menso así (usando formato de f-string para ejemplificar nada más)

- Total entries: {T}
- Entries that need work: {F + U} ({(F + U)/T * 100:.2f} %)
  - Fuzzy: {F}
  - Untranslated: {U}

Dime qué te parece la idea.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

De acuerdo en que son confusas.
Mejorado acá 1915095 y luego arreglado acá 9938321

- Percent translated: {pofile_percent_translated}%
- Entries: {pofile_entries}
- Untranslated: {pofile_untranslated}

if pofile.fuzzy == 0 and any([
pofile.translated_nb == pofile.po_file_size,
pofile.untranslated_nb == 0,
]):
print(f'Skipping {pofilename}. The file is 100% translated already.')
sys.exit(1)
Please, comment here if you want this file to be assigned to you and a member will assign it to you as soon as possible, so you can start working on it.

# https://pygithub.readthedocs.io/en/latest/github_objects/Repository.html#github.Repository.Repository.create_issue
title = f'Translate `{pofilename}`'
urlfile = pofilename.replace('.po', '.html')
issue = repo.create_issue(
title=title,
body=f'''This needs to reach 100% translated.
Remember to follow the steps in our [Contributing Guide](https://python-docs-es.readthedocs.io/page/CONTRIBUTING.html).'''

The rendered version of this file will be available at https://docs.python.org/es/3.8/{urlfile} once translated.
Meanwhile, the English version is shown.

Current stats for `{pofilename}`:
class IssueAlreadyExistingError(Exception):
"""Issue already existing in GitHub"""

- Fuzzy: {pofile.fuzzy_nb}
- Percent translated: {pofile.percent_translated}%
- Entries: {pofile.translated_nb} / {pofile.po_file_size}
- Untranslated: {pofile.untranslated_nb}

Please, comment here if you want this file to be assigned to you and a member will assign it to you as soon as possible, so you can start working on it.
class PoFileAlreadyTranslated(Exception):
"""Given PO file is already 100% translated"""



def check_issue_not_already_existing(pofilename):
issues = repo.get_issues(state='open')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Supongo que no es un problema, pero sería bueno revisar cómo cachar esta request, veo que nos traemos todos los issues y supongo esto ocurre para cada pofile.

Copy link
Collaborator

@rtobar rtobar Nov 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totalmente de acuerdo con el comentario, no creo que sea mucho cambiar el código para hacer una sola query?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done por acá: 6916e38

for issue in issues:
if pofilename in issue.title:

print(f'Skipping {pofilename}. There is a similar issue already created at {issue.html_url}')
raise IssueAlreadyExistingError()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: no hace falta instanciar acá

Suggested change
raise IssueAlreadyExistingError()
raise IssueAlreadyExistingError



def check_translation_is_pending(pofile):
if pofile.fuzzy == 0 and any([
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Veo que este any estaba en el original, pero lo encuentro innecesariamente confuso. En vez de hacer any([a, b]) puedes hacer (a or b).

Pero creo que la lógica podría ser simplificada de todas formas. Se me hace que if not pofile.fuzzy and not pofile.untranslated sería suficiente, pero habría que probar

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

De acuerdo, mejoré un poco esto acá: 9461867

pofile.translated == pofile.entries,
pofile.untranslated == 0,
]):
print(f'Skipping {pofile.filename}. The file is 100% translated already.')
raise PoFileAlreadyTranslated()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ni acá




def issue_generator(pofilename):
pofile = PoFileStats(Path(pofilename))

check_issue_not_already_existing(pofilename)
check_translation_is_pending(pofile)

urlfile = pofilename.replace('.po', '.html')
title = ISSUE_TITLE.format(pofilename=pofilename)
body = ISSUE_BODY.format(
python_version=PYTHON_VERSION,
urlfile=urlfile,
pofilename=pofilename,
pofile_fuzzy=pofile.fuzzy,
pofile_percent_translated=pofile.percent_translated,
pofile_entries=pofile.entries,
pofile_untranslated=pofile.untranslated,
)
# https://pygithub.readthedocs.io/en/latest/github_objects/Repository.html#github.Repository.Repository.create_issue
issue = repo.create_issue(title=title, body=body, labels=ISSUE_LABELS)

return issue

def create_issues(only_one=False):
po_files = glob("**/*.po")
existing_issue_counter = 0
already_translated_counter = 0
created_issues_counter = 0

print(f"TOTAL PO FILES: {len(po_files)}")

for pofilename in po_files:
try:
issue = issue_generator(pofilename)
created_issues_counter += 1
print(f'Issue "{issue.title}" created at {issue.html_url}')
if only_one:
break
except IssueAlreadyExistingError:
existing_issue_counter += 1
except PoFileAlreadyTranslated:
already_translated_counter += 1

print("Stats:")
print(f"- Existing issues: {existing_issue_counter}")
print(f"- Already translated files: {already_translated_counter}")
print(f"- Created issues: {created_issues_counter}")




def main():
error_msg = "Specify PO filename or '--all' to create all the issues, or '--one' to create the next one issue"
if len(sys.argv) != 2:
raise Exception(error_msg)

arg = sys.argv[1]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Una idea que tenía era agregar un modo "dry run", pero queda para el futuro, necesitaría más cambios de los que hay hasta el momento

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

de acuerdo con la idea, y con que se haga a futuro 😅


if arg == "--all":
create_issues()

elif arg == "--one":
create_issues(only_one=True)

else:
try:
issue_generator(arg)
except FileNotFoundError:
raise Exception(error_msg)

Remember to follow the steps in our [Contributing Guide](https://python-docs-es.readthedocs.io/page/CONTRIBUTING.html).''',
)
print(f'Issue "{title}" created at {issue.html_url}')
if __name__ == "__main__":
main()