A lack of seriousness in the framework evolution for some time?

Hi everyone,

It’s been a while since I didn’t post here. Since last summer period has been very busy for me because of the end of my studies and various works.

As some of you have heard me say, I claim to be a “functional consultant”, that is to say:

-Able to understand and gather the needs of the user company
-With this understanding and that of the main modules of OpenERP, I’m supposed to find the most sustainable way to develop a feature before sending the specification to the development.
-I am able to do basic developments, such as adding a new field, adapting a view, a right access, correct translation …. Anything that is supported by the framework as it is supposed to manage the most common needs of the ERPs. This allows to submit the interface to the customer early in the project which is a highly effective asset to save time on the project.
-I am not able to make pure Python development. I am still able to read and sometimes make minor corrections.

Why I am making this definition of functional consultant? Because I think anyone who has business expertise, can become a functional consultant on OpenERP, it will just take time.

This is possible because from the beginning, OpenERP has established the framework OpenObject that manages database objects, menus, views, access rights, etc. … Making feasible to just anyone to make key changes on the ERP’s process. This has always been the main strengh of OpenERP! Ease of access to code features

Ok Yannick, we already know all this, what do you mean?

I mean that I feel since the relocation of development teams in India this spirit has greatly lost.

I’ll show you an example that I was literally stunned the day I saw that, without finding the time to talk about it before today.
Here is the code for the statistical analysis of invoices:

def init(self, cr):
        tools.drop_view_if_exists(cr, 'account_invoice_report')
        cr.execute("""
            create or replace view account_invoice_report as (
                 select min(ail.id) as id,
                    ai.date_invoice as date,
                    to_char(ai.date_invoice, 'YYYY') as year,
                    to_char(ai.date_invoice, 'MM') as month,
                    to_char(ai.date_invoice, 'YYYY-MM-DD') as day,
                    ail.product_id,
                    ai.partner_id as partner_id,
                    ai.payment_term as payment_term,
                    ai.period_id as period_id,
                    (case when u.uom_type not in ('reference') then
                        (select name from product_uom where uom_type='reference' and active and category_id=u.category_id LIMIT 1)
                    else
                        u.name
                    end) as uom_name,
                    ai.currency_id as currency_id,
                    ai.journal_id as journal_id,
                    ai.fiscal_position as fiscal_position,
                    ai.user_id as user_id,
                    ai.company_id as company_id,
                    count(ail.*) as nbr,
                    ai.type as type,
                    ai.state,
                    pt.categ_id,
                    ai.date_due as date_due,
                    ai.address_contact_id as address_contact_id,
                    ai.address_invoice_id as address_invoice_id,
                    ai.account_id as account_id,
                    ai.partner_bank_id as partner_bank_id,
                    sum(case when ai.type in ('out_refund','in_invoice') then
                         ail.quantity / u.factor * -1
                        else
                         ail.quantity / u.factor
                        end) as product_qty,
                    sum(case when ai.type in ('out_refund','in_invoice') then
                         ail.quantity*ail.price_unit * -1
                        else
                         ail.quantity*ail.price_unit
                        end) / cr.rate as price_total,
                    sum(case when ai.type in ('out_refund','in_invoice') then
                         ai.amount_total * -1
                        else
                         ai.amount_total
                         end) / (CASE WHEN
                              (select count(l.id) from account_invoice_line as l
                               left join account_invoice as a ON (a.id=l.invoice_id)
                               where a.id=ai.id) <> 0
                            THEN
                              (select count(l.id) from account_invoice_line as l
                               left join account_invoice as a ON (a.id=l.invoice_id)
                               where a.id=ai.id)
                            ELSE 1
                            END) / cr.rate as price_total_tax,
                    (case when ai.type in ('out_refund','in_invoice') then
                      sum(ail.quantity*ail.price_unit*-1)
                    else
                      sum(ail.quantity*ail.price_unit)
                    end) / (CASE WHEN
                         (case when ai.type in ('out_refund','in_invoice')
                          then sum(ail.quantity/u.factor*-1)
                          else sum(ail.quantity/u.factor) end) <> 0
                       THEN
                         (case when ai.type in ('out_refund','in_invoice')
                          then sum(ail.quantity/u.factor*-1)
                          else sum(ail.quantity/u.factor) end)
                       ELSE 1
                       END)
                     / cr.rate as price_average,

                    cr.rate as currency_rate,
                    sum((select extract(epoch from avg(date_trunc('day',aml.date_created)-date_trunc('day',l.create_date)))/(24*60*60)::decimal(16,2)
                        from account_move_line as aml
                        left join account_invoice as a ON (a.move_id=aml.move_id)
                        left join account_invoice_line as l ON (a.id=l.invoice_id)
                        where a.id=ai.id)) as delay_to_pay,
                    sum((select extract(epoch from avg(date_trunc('day',a.date_due)-date_trunc('day',a.date_invoice)))/(24*60*60)::decimal(16,2)
                        from account_move_line as aml
                        left join account_invoice as a ON (a.move_id=aml.move_id)
                        left join account_invoice_line as l ON (a.id=l.invoice_id)
                        where a.id=ai.id)) as due_delay,
                    (case when ai.type in ('out_refund','in_invoice') then
                      ai.residual * -1
                    else
                      ai.residual
                    end)/ (CASE WHEN
                        (select count(l.id) from account_invoice_line as l
                         left join account_invoice as a ON (a.id=l.invoice_id)
                         where a.id=ai.id) <> 0
                       THEN
                        (select count(l.id) from account_invoice_line as l
                         left join account_invoice as a ON (a.id=l.invoice_id)
                         where a.id=ai.id)
                       ELSE 1
                       END) / cr.rate as residual
                from account_invoice_line as ail
                left join account_invoice as ai ON (ai.id=ail.invoice_id)
                left join product_template pt on (pt.id=ail.product_id)
                left join product_uom u on (u.id=ail.uos_id),
                res_currency_rate cr
                where cr.id in (select id from res_currency_rate cr2  where (cr2.currency_id = ai.currency_id)
                and ((ai.date_invoice is not null and cr.name <= ai.date_invoice) or (ai.date_invoice is null and cr.name <= NOW())) limit 1)
                group by ail.product_id,
                    ai.date_invoice,
                    ai.id,
                    cr.rate,
                    to_char(ai.date_invoice, 'YYYY'),
                    to_char(ai.date_invoice, 'MM'),
                    to_char(ai.date_invoice, 'YYYY-MM-DD'),
                    ai.partner_id,
                    ai.payment_term,
                    ai.period_id,
                    u.name,
                    ai.currency_id,
                    ai.journal_id,
                    ai.fiscal_position,
                    ai.user_id,
                    ai.company_id,
                    ai.type,
                    ai.state,
                    pt.categ_id,
                    ai.date_due,
                    ai.address_contact_id,
                    ai.address_invoice_id,
                    ai.account_id,
                    ai.partner_bank_id,
                    ai.residual,
                    ai.amount_total,
                    u.uom_type,
                    u.category_id
            )
        """)

134 lines of pure SQL query, just to create a statistical analysis …

Ok then it may be necessary, but it is in the framework we should find this code, not in one of the functional modules! Here we should just tell him that’s the name of the object, these are the fields to be analyzed and go on. I do not recognize any of the OpenERP code usually ultra-simple, this code just look like it was done without any forethought.What effect? What happen if me, a simple functional consultant,  I just want to add an additional field to analyze? Or create a new statistical analysis on another object?  I can’t, and so I have to send this to developers who will rewrite theses 134lines (which is a problem even worse, I come back).In recent years it seems there was no serious efforts in the framework improvement. Ok then I support 300% web client, payroll and even the POS. But I begin to see big problems in the future for OpenERP that could collapse on its foundation, that is to say the framework.

To be clear, I’m not saying that OpenERP is poorly written, it is not “pythonic” or another. I do not have the skills for that and others are already doing it (dedication to the Tryton people which I am sure must enjoy this post …).
What I mean is that the functional should be able to change a lot more features than now on OpenERP.

What happens when one places an order from one status to another, such as sending an email, can add a module in a configuration wizard, or transfer the value of the field that you just created in sale.order to his account.invoice etc. …
There are still lots of things we can do, manage by OpenObject even more prominent roles in the ERP so we limit the amount of Python code to a minimum.

Moreover, I think this will be a good way to limit the amount of code that is laid each day by the Indian teams. I am sorry to say this but a lot of places I sometimes feel, pardon the expression, they just “pissed code” as if they were paid to the line. I am very concerned that this is a big risk and that eventually we could  no longer maintain the software later.
Crop development teams in OpenObject developments is probably the easier way to make sure there is only one way to make the work done, the simpliest and the better. This will eventually be able to more easily maintain the software or especially deep module redesign.

And moreover for some features they are already managed by OpenObject. For example creating a new invoice with the values ​​of a purchase order via ir.actions.server. Unfortunately, as certified module not use it, nobody uses these features …

In short, let me be clear on what I suggest: First, rewrite all the modules so they use ir.actions.server or an equivalent system. Then continue to improve the framework until the Python code in the modules is reduced to a minimum.

I am sure that most readers will rebel against this idea, too much work, you should leave some functions in python to have more flexibility etc. … No worries, I just try to start the debate and I  just hope it will lead to interesting conclusions.

I would however point the finger on one last problem, I think the worst, and that can be solved just by extending features of OpenObject.

Take a Partner A highly experienced and top contributors. He developed a module that is widely used by other community members.

In his module, he created a field in X and sale.order account.invoice, and he made sure that its value is passed to its sale.order account.invoice.

Here is the code he had to do:

    def _make_invoice(self, cr, uid, order, lines, context=None):
        journal_obj = self.pool.get('account.journal')
        inv_obj = self.pool.get('account.invoice')
        obj_invoice_line = self.pool.get('account.invoice.line')
        if context is None:
            context = {}

        journal_ids = journal_obj.search(cr, uid, [('type', '=', 'sale'), ('company_id', '=', order.company_id.id)], limit=1)
        if not journal_ids:
            raise osv.except_osv(_('Error !'),
                _('There is no sales journal defined for this company: "%s" (id:%d)') % (order.company_id.name, order.company_id.id))
        a = order.partner_id.property_account_receivable.id
        pay_term = order.payment_term and order.payment_term.id or False
        invoiced_sale_line_ids = self.pool.get('sale.order.line').search(cr, uid, [('order_id', '=', order.id), ('invoiced', '=', True)], context=context)
        from_line_invoice_ids = []
        for invoiced_sale_line_id in self.pool.get('sale.order.line').browse(cr, uid, invoiced_sale_line_ids, context=context):
            for invoice_line_id in invoiced_sale_line_id.invoice_lines:
                if invoice_line_id.invoice_id.id not in from_line_invoice_ids:
                    from_line_invoice_ids.append(invoice_line_id.invoice_id.id)
        for preinv in order.invoice_ids:
            if preinv.state not in ('cancel',) and preinv.id not in from_line_invoice_ids:
                for preline in preinv.invoice_line:
                    inv_line_id = obj_invoice_line.copy(cr, uid, preline.id, {'invoice_id': False, 'price_unit': -preline.price_unit})
                    lines.append(inv_line_id)
        inv = {
            'name': order.client_order_ref or '',
            'origin': order.name,
            'type': 'out_invoice',
            'reference': order.client_order_ref or order.name,
            'account_id': a,
            'partner_id': order.partner_id.id,
            'journal_id': journal_ids[0],
            'address_invoice_id': order.partner_invoice_id.id,
            'address_contact_id': order.partner_order_id.id,
            'invoice_line': [(6, 0, lines)],
            'currency_id': order.pricelist_id.currency_id.id,
            'comment': order.note,
            'payment_term': pay_term,
            'fiscal_position': order.fiscal_position.id or order.partner_id.property_account_position.id,
            'date_invoice': context.get('date_invoice',False),
            'company_id': order.company_id.id,
            'user_id': order.user_id and order.user_id.id or False,
            'champ_x': order.champ_x
        }
        inv.update(self._inv_get(cr, uid, order))
        inv_id = inv_obj.create(cr, uid, inv, context=context)
        data = inv_obj.onchange_payment_term_date_invoice(cr, uid, [inv_id], pay_term, time.strftime('%Y-%m-%d'))
        if data.get('value', False):
            inv_obj.write(cr, uid, [inv_id], data['value'], context=context)
        inv_obj.button_compute(cr, uid, [inv_id])
        return inv_id
50 lines, when in fact the only line that he really coded is in red here. And yet we were lucky, the original function could have been much longer.
As it is not possible via the module to insert the code directly, he had to copy and paste the entire function to add this little line.And the real problem happens. No chance the OpenERP devs change after the original function and other functions elsewhere, so that the former causes a bug now. Our partner A must now repeat the copy and paste of the original function if it does not want his module to be considered buggy.This forces our partner to monitor all its modules, just to check that OpenERP SA didn’t modify the original function. And since the functions are becoming longer, the risk of the bugs are all the more. Finally, the ultimate case, imagine if a partner created a module B that inherits from the function module A … What I mean by that is that it is the entire ecosystem of community modules that is unstable due to this overuse of Python functions that are not AT ALL suitable for a modular system as OpenERP.

I think we’re right into it today, we see that more and more quality modules arrive on OpenERP, but they are buggy as soon as OpenERP SA made changes in the functions of certified modules, which themselves become increasingly long and complex.

We must at all costs find another solution for module developers to avoid having to copy the original function when they just want to add some details as a field to be transferred. And for me the only viable solution, as it is clear that we can not do that directly in the Python code, is to extend the OpenObject capabilities.

Thank you for taking the reading. I express my personal conviction, which is somewhat isolated. I am not a developer, so if I may say big shit, please tell me in comments.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>