Skip to content

Update benchmark #36

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

Update benchmark #36

wants to merge 5 commits into from

Conversation

vvo
Copy link

@vvo vvo commented Oct 12, 2014

Hello, I updated all TODO MVC examples and of the benchmark BUT the vuejs one.

I will let you update the vuejs example if you agree.

I added a script to update all frameworks (e542837)

I ran the benchmarks again and here are the results:

100 items:

{
    "Vue": 192.28999999904772,
    "Backbone": 308.4640000015497,
    "Knockout": 8655.599000005168,
    "Ember": 1397.6430000002438,
    "Angular": 11847.705000000133,
    "React": 54.93700000079116,
    "Om": 43.8800000010815,
    "Ractive": 881.4449999990757
}

1000 items:

{
    "Vue": 1211.6989999958605,
    "Backbone": 4051.5769999983604,
    "Knockout": 9586.620000001858,
    "Ember": 21326.647000001685,
    "Angular": 45698.4569999986,
    "React": 194.11399999808054,
    "Om": 176.4690000054543,
    "Ractive": 55959.131999996316
}

I am not a pro-angular/react etc, only wanted to get some fresh numbers.

What do you think?

@yyx990803
Copy link
Member

Hey man, thanks a lot for the updates. However, based on some discussion with other framework authors (guys from Ember, React, Ractive etc.) the consensus is that this benchmark doesn't provide meaningful insights anymore because of the following:

  1. The "synchrorously trigger an action xxx times" doesn't reflect real world user actions; The original intention of these benchmarks were for comparing Browser performance rather than that of frameworks.
  2. Due to internal implementation differences, frameworks that uses async rendering (e.g. Vue) gains the advantage by skipping part of the calculations that happened in the same event loop. The real world user experience doesn't demonstrate such dramatic difference. Although Vue is decently fast even without async rendering, the benchmark is to some extent misleading.
  3. Overall this benchmark suite gave rise to more controversy and hostility than constructive feedback, so I'm removing this for the 0.11 doc site and try to find another way to accurately measure the performance.

P.S. React's 0.11 update seems to break the old test routine because of changes to its synthetic event system. In handleNewTodoKeyDown the event.which is undefined, so no todos were ever added.

@yyx990803 yyx990803 closed this Oct 12, 2014
@vvo
Copy link
Author

vvo commented Oct 12, 2014

Allright I knew it would be possible that you would not accept this.

I understand your points, thanks!

Looking forward to the new 0.11 website so.

@vvo vvo deleted the update-benchmark branch October 12, 2014 18:39
dingyiming added a commit that referenced this pull request Dec 21, 2016
maks-rafalko pushed a commit to infection/site that referenced this pull request Jun 29, 2017
lex111 pushed a commit to lex111/vuejs.org that referenced this pull request Feb 3, 2018
Найденные исправления
landry-some added a commit to landry-some/site that referenced this pull request Oct 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants