Size: 5501
Comment:
|
Size: 3410
Comment:
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
##master-page:HomepageTemplate #format wiki #language en Email: <<MailTo(in3xes AT SPAMFREE gmail DOT com)>> |
## page was renamed from in3xes ## page was renamed from Pradeepkumar == Google Summer of Code-2010 == ---- === Contact Details: === {{{ Name: Pradeepkumar Gayam Email: in3xes@gmail.com IRC Nick: in3xes (freenode.net) Jabber ID: in3xes [AT] gmail.com (talk.google.com) }}} My GSoC [[http://bitbucket.org/in3xes/gsoc/src/tip/application|application]] on bitbucket. |
Line 6: | Line 14: |
IRC Nick: inXs_ | ---- ''' ''' |
Line 8: | Line 17: |
'''About Me:''' | === Journal: === 2010-06-23: Study shallow clone how they are related to parent delta |
Line 10: | Line 20: |
- Student at [[http://www.iitr.ac.in|IIT]] in India. Mathematics is my major. | 2010-06-24: Revlog experiments to improve performance |
Line 12: | Line 22: |
- Fan of C, Python, Haskell. You can find some of my small projects on [[http://www.github.com/in3Xes|Github]]. - | 2010-06-25: More experiments, fixed the issue, so that it will not differ for old revlogs, still expriments needs to be done. |
Line 14: | Line 24: |
- Love solving mathematics problems by programming. [[http://www.github.com/in3Xes/projecteuler|Solutions]] for the problems in [[http://www.projecteuler.net|Project Euler]]. Solved in python. | 2010-07-26: Study wire-proto, understand how bundle, unbundle is being done. |
Line 16: | Line 26: |
- Experienced user of git. Comfortable with Bazaar. | 2010-06-27: Week end |
Line 18: | Line 28: |
- Find me on [[http://www.twitter.com/in3Xes|Twitter]] | ---- === Work Progress: === ''' '''Few things related revlog: |
Line 20: | Line 32: |
== Google Summer of Code-2010 == I am interested in "Parent delta". I can also work on conversion tools. But I am mainly focusing on Parent Delta Plan. |
1) A [[http://markmail.org/download.xqy?id=ooqvjdaor4t3sxan&number=1|script]] by tonfa, proof for parent delta. |
Line 23: | Line 34: |
A word about project: | 2) mpm [[http://markmail.org/thread/ooqvjdaor4t3sxan|explaining]] how space inefficiency is caused. |
Line 25: | Line 36: |
Mercurial calculates diffs against previous revision rather than parent. In some cases this implementation is space inefficient, and it is more sensible to store deltas against parent. This project is about implementing [[http://mercurial.selenic.com/wiki/ParentDeltaPlan|Parent Delta Plan]]. This implementation changes the structure of revlogs, so wire protocol has to be extended to allow backward compatibility. | 3) A [[http://mercurial.selenic.com/hg/hg/file/45aabc523c86/contrib/shrink-revlog.py|python script]] in mercurial repo to shrink revlog by sorting in topological order. |
Line 27: | Line 38: |
============================================================================ | Earlier in this week I started working on revlog, most of the work is already done by tonfa. He shared patches. All I did was understand what he did, and rewrite those patches with minor changes. |
Line 29: | Line 40: |
=== Proposal === TO DO(Tentative): Major: |
'''strategy:''' |
Line 32: | Line 42: |
* Changes in revlog * Changes in wire protocol |
As mpm suggested, I have changed the way new revision is being added to existing revlog, and make corresponding changes in creating a given revision of file. The crucial step (performance affecting step) is creating a given version of file. previously to create a given version of a file first we take base revision , to that we keep adding the deltas till given revision. Earlier deltas are added in linear manner. So creating a given revision of file is simply seeking O(1) times. |
Line 35: | Line 44: |
Changes in revlog: | ''Defination of base in revlog is not given anywhere. From revlog, a base is the version where we add full text of file to the revlog instead of delta.'' |
Line 37: | Line 46: |
. Change revlog structure, now it appends the next delta to existing data.This helps in reconstrfucting a revision at O(1) seeks. If | example scenario: |
Line 39: | Line 48: |
parent delta is implemented better compression can be achieved but, may have to compromise with number of seeks. | We want to create a R1 revision of a file. The parents of R1 are P1 and P2. Base of R1 is B1. |
Line 41: | Line 50: |
* Current:Linear delta model . --> O(1) seeks to create any revision. * Parent delta model * ** I think this can be completed quickly, tonfa shared patches for this*** . --> An intelligent algorithm to reduce number of seeks to reconstruct any revision.I think constructing revision is |
Consider a situation where P2 is null, and P1 < B1. In this case I simply can't take B1 and create R1. I have first find base and parents of P1 and construct P1. The same case might repeat. where parent of P1 is < base of P1. So, first I have to find a base which is ancestor of all the parents, then create a chain of deltas from that base to construct a given revision. |
Line 47: | Line 52: |
. not straight forward as it is now. It's not going to be linear. . --> Investigation about merge revision. --> Optimization, and investigation for better implementation(If possible). --> Continuous parallel testing. |
Output of --profile says that, what ever the decrease in the performance is caused only by this function which used for constructing the delta chain. So, I need to find a better algorithm to construct the delta chain. The current algorithm for constructing the delta chain is. |
Line 50: | Line 54: |
Changes in wire protocol: | {{{ def deltachain(self, rev): chain = [] while self.base(rev) != rev: chain.append(rev) rev = self.deltaparent(rev) chain.reverse() return rev, chain |
Line 52: | Line 63: |
* Changes in wire protocol to allow backward computability. * Implementation of changes according to old, new servers,client, repos. . --> Backward computability matrix (by tonfa): http://mercurial.selenic.com/wiki/ParentDeltaPlan . --> There are things to be discussed, and learned before implementation. * Rigorous testing * Integrating with other's work.(Changeset discovery, Lightweight copies/renames) About Me: . I am student at Indian Institute of Technology, Roorkee. I am currently pursing third year of 5 years degree, Mathematics as major. I am a Linux enthusiast. Interested in open source. I have been learning C,Python. Some of my projects can be found on github ( http://www.github.com/in3Xes ). I have never got opportunity to contribute to open source. GSoC would be stepping stone for that. . I am project leader in IMG ( Information Management Group, IIT Roorkee, http://www.iitr.ac.in/IMG/ ), a group which manages intranet, . student databases, institute website. I have worked on PHP, Java in web development. Contact Information: . Name: Pradeepkumar Gayam Email: in3xes@gmail.com . IRC Nick: inXs_ (freenode.net) . Jabber ID: in3xes [AT] gmail.com (talk.google.com) Schedule: . I would start coding well before the schedule given in the time line. My final exams will end by May-15th. I can start working . rigorously shortly after exams. . Till May-15th(or 24th according to time line) I will be reading code, documentation, getting used to coding style, getting ready to start coding, little experimentation and stuff. '''Timeline''' ''Changes in revlog structure'': [Total 3 weeks] * Parent delta model: . --> An intelligent algorithm to reduce number of seeks to reconstruct any revision.I think constructing revision is not . straight forward as it is now. It's not going to be linear. . [(It can take 1 1/2 to 2 weeks to bring it to working stage, Roughly around 5th-Jun)] . --> Investigation about merge revision. --> Optimization, and investigation for better implementation(If possible). . --> Continuous parallel testing. . [( I would finish testing and other work nearly in 10 days, by 15th-Jun)] ''Changes in wire protocol'':[Total 3 to 3 1/2 weeks] * Changes in wire protocol to allow backward computability. . [(For midterm evaluation, this may not be complete, but significant part of it)] . --> Implementation of changes according to old, new servers,client, repos. . >> Backward compatibility matrix (by tonfa): http://mercurial.selenic.com/wiki/ParentDeltaPlan . >> There are things to be discussed, and learned before implementation. * Rigorous testing. . [(I will start testing shortly after mid-term evaluation, i.e., around 18th-July, would take a week to complete)] * Integrating with other's work. . [(Changeset discovery, Lightweight copies/renames). ( Remaining time, depending upon other students)] Link to my GSoC [[http://www.bitbucket.org/in3xes/gsoc|application]] on bitbucket.org ## You can even more obfuscate your email address by adding more uppercase letters followed by a leading and trailing blank. |
def deltaparent(self, rev): if self.base(rev) == rev: return nullrev elif self.flags(rev) & REVLOG_PARENTDELTA_FLAGS: return self.parentrevs(rev)[0] else: return rev- }}} |
Google Summer of Code-2010
Contact Details:
Name: Pradeepkumar Gayam Email: in3xes@gmail.com IRC Nick: in3xes (freenode.net) Jabber ID: in3xes [AT] gmail.com (talk.google.com)
My GSoC application on bitbucket.
Journal:
2010-06-23: Study shallow clone how they are related to parent delta
2010-06-24: Revlog experiments to improve performance
2010-06-25: More experiments, fixed the issue, so that it will not differ for old revlogs, still expriments needs to be done.
2010-07-26: Study wire-proto, understand how bundle, unbundle is being done.
2010-06-27: Week end
Work Progress:
Few things related revlog:
1) A script by tonfa, proof for parent delta.
2) mpm explaining how space inefficiency is caused.
3) A python script in mercurial repo to shrink revlog by sorting in topological order.
Earlier in this week I started working on revlog, most of the work is already done by tonfa. He shared patches. All I did was understand what he did, and rewrite those patches with minor changes.
strategy:
As mpm suggested, I have changed the way new revision is being added to existing revlog, and make corresponding changes in creating a given revision of file. The crucial step (performance affecting step) is creating a given version of file. previously to create a given version of a file first we take base revision , to that we keep adding the deltas till given revision. Earlier deltas are added in linear manner. So creating a given revision of file is simply seeking O(1) times.
Defination of base in revlog is not given anywhere. From revlog, a base is the version where we add full text of file to the revlog instead of delta.
example scenario:
We want to create a R1 revision of a file. The parents of R1 are P1 and P2. Base of R1 is B1.
Consider a situation where P2 is null, and P1 < B1. In this case I simply can't take B1 and create R1. I have first find base and parents of P1 and construct P1. The same case might repeat. where parent of P1 is < base of P1. So, first I have to find a base which is ancestor of all the parents, then create a chain of deltas from that base to construct a given revision.
Output of --profile says that, what ever the decrease in the performance is caused only by this function which used for constructing the delta chain. So, I need to find a better algorithm to construct the delta chain. The current algorithm for constructing the delta chain is.
def deltachain(self, rev): chain = [] while self.base(rev) != rev: chain.append(rev) rev = self.deltaparent(rev) chain.reverse() return rev, chain def deltaparent(self, rev): if self.base(rev) == rev: return nullrev elif self.flags(rev) & REVLOG_PARENTDELTA_FLAGS: return self.parentrevs(rev)[0] else: return rev-