ISSN 0798 1015
ISSN 0798 1015
Vol. 38 (Nº 49) Year 2017. Page 21
Evgeniya V. UFIMTSEVA 1; Yusup Yu. GALYAMOV 2; Yulia V. PODOPRIGORA 3; Nikolay R. SHADEYKO 4; Alexey A. SELIVERSTOV 5
Received: 12/06/2017 • Approved: 20/07/2017
ABSTRACT: Nowadays, the problems of efficiency in any field of human activity attract much attention at the stage of determination of the reasonability of goals, tasks, measures, and also while selecting the indexes reflecting its achievement. The urgency of the research is determined by the theoretical and practical significance of the formation of approaches to the assessment of implementation of budget programs to improve the efficiency of budget costs and social and economic development of the territory. The topic of the research is stipulated by the vector in the field of budgetary policy specified in the Budget Message of the President of the Russian Federation, including the development of the program of improvement of the efficiency of the control of public finances for the period up to 2018. The essence of the scientific problem results from the necessity to review and supplement the methodological provisions of the assessment of the efficiency of implementation of budget programs regarding the problems of efficiency of budget costs and the level of achievement of the expected effects as a result of the implementation of the planned actions. In this regard, the purpose of the research is to analyze the methodological aspects of the assessment of the efficiency of implementation of budget programs and also their structure and content on the base of the study of some regulatory acts of the municipalities that establish the order of assessment of implementation of budget programs. The research considers the methodical tools of assessment of the efficiency of budget costs; the analysis of the methodical aspects of the assessment of the efficiency of budget programs of municipalities was performed. The research was performed on the base of the methods of comparative and analytical, terminological, logical, expert, and system analysis and also the theoretical research. In this work, it was concluded that the regulatory acts of the municipalities that establish the procedure of assessing the efficiency of implementation of budget programs require the rework (correction) and in some cases the critical changes should be made in the methodical part. |
RESUMEN: Hoy en día, los problemas de eficiencia en cualquier campo de la actividad humana atraen mucha atención en la etapa de determinación de la razonabilidad de metas, tareas, medidas, y también mientras seleccionamos los índices que reflejan su logro. La urgencia de la investigación está determinada por la importancia teórica y práctica de la formación de enfoques para la evaluación de la ejecución de los programas presupuestarios para mejorar la eficiencia de los costos presupuestarios y el desarrollo social y económico de la territorio. El tema de la investigación está estipulado por el vector en el ámbito de la política presupuestaria especificada en el mensaje presupuestario del Presidente de la Federación de Rusia, incluyendo el desarrollo del programa de mejoramiento de la eficiencia del control público Finanzas para el período hasta 2018. La esencia del problema científico resulta de la necesidad de revisar y complementar las disposiciones metodológicas de la evaluación de la eficacia de la ejecución de los programas presupuestarios con respecto a los problemas de eficiencia de los costes presupuestarios y el nivel de logro de los efectos previstos como resultado de la implementación de las acciones planificadas. A este respecto, el objetivo de la investigación es analizar los aspectos metodológicos de la evaluación de la eficiencia de la implementación de los programas presupuestarios y también su estructura y contenido sobre la base del estudio de algunos actos regulatorios de los municipios que establecen el orden de evaluación de la implementación de programas presupuestarios. La investigación considera las herramientas metódicas de evaluación de la eficiencia de los costos presupuestarios; el análisis de los aspectos metódicos del gravamen de la eficacia de los programas del presupuesto de municipios fue realizado. La investigación se realizó sobre la base de los métodos de análisis comparativo y analítico, terminológica, lógico, experto, y del sistema, así como de la investigación teórica. En este trabajo se concluye que los actos regulatorios de los municipios que establecen el procedimiento de evaluación de la eficiencia de la ejecución de los programas presupuestarios requieren la reintervención (corrección) y en algunos casos los cambios críticos deben hacerse en el parte metódica. |
The interests of citizens of the Russian Federation, their material wealth, level of consumption, living standard, etc. depend completely upon the social and economic reforms and the development of the economy of the country that have taken place during the last decade. In this regard, the efficient control of public finance favors the increase in the level and living standard of the citizens, economic growth, modernization of the economy and social sphere and also the achievement of other goals of the social and economic development of Russia.
The existence of some problems of control of the public finance has become the reason for the President of the Russian Federation to specify the necessity to develop the programs of improvement of the efficiency of control of the public finance in the Budget Message. It should be mentioned that the achievement of efficiency of budget costs in the modern condition is one of the priority directions of the state and municipal policy in the Russian Federation. Under the budget restrictions, the fact of achievement of the efficiency (economic, financial, social, infrastructural, ecological, etc.) is of great importance.
Nowadays, when estimating the efficiency of implementation of budget programs, the authorities of the state and municipal financial control shift their attention to the problems of the intended use of the public funds for the control of achievement of the planned results and reasonability of the costs made. According to the information above, the financial provision of the budget measures (programs) is performed mainly by the methods "from the achieved results". Moreover, the methodological instructions of assessment of the efficiency of budget programs are mostly formal financial feasibility studies for the projects of solutions. In this regard, the urgency of the specified scientific problem stipulates the importance of the present research.
The analysis of modern studies allowed determining the group of foreign and domestic scientists who were engaged in study of the particular aspects of the said scientific problem. Thus, the study of problems of the efficiency of activity was paid much attention in the researchers’ works.
Before we turn to the problems of the efficiency of activity, we will consider the main discrepancies of the categories "efficiency" and "effectiveness" that are offered by different authors because often the efficiency is associated with effect that is the result of impact of something on the object of research (Table 1).
Table 1
Main discrepancies of the categories "efficiency" and "effectiveness"
Author |
Efficiency |
Effectiveness |
E.G. Dolan (Dolan, 1994) |
Choice of right purposes on which all energy is focused |
Achievement of the set goals with minimal costs and errors. The effectiveness without efficiency can lead to the bankruptcy of an enterprise as any activity has a result that does not have to be positive |
M.H. Mescon, M. Albert, F. Hedouri (Mescon, 2007) |
Internal economic efficiency that measures the best use of resources |
It is the most complete characteristic of the enterprise, specifying the economic efficiency as well as the interrelation with the external environment and they determine the effectiveness as the correlation between the volume and costs and also as a category that presupposes the control of the additional parameters (fulfilment of obligations, adaptability, development) |
V.V. Kovalev (Kovalev, 2007) |
Relative index, comparing the obtained effect with the costs or resources, used for the achievement of this effect |
Value reflecting the profitability, dynamic character, goal fulfilment, efficiency of use of the economic potential, position in the securities market |
Yu.V. Perevalov, I.E. Gimadi, V.V. Dobrodey (Perevalov et al., 1997) |
Heterogeneous notion that should be considered in two aspects: effectiveness and economic efficiency |
Degree of achievement of main goals that is characterized by the quantitative values that in turn reflect the goals regardless of the efficiency of their fulfilment |
I.I. Mazur, V.D. Shapiro (Mazur et al., 2003) |
It is always a particular correlation (result with goals or result with costs for its obtaining), that is, a relative value; this category has a managerial character and reflects, first of all, the degree of achievement of the set goals |
|
Thus, the generalization of various points of view of researchers regarding the determination of the terms "effectiveness" and "efficiency" allows concluding that these notions are not identical.
Then we will address to the urgent and rather complicated problems of criteria and values of the efficiency of activity that become an object of the wide range of scientific studies and discussions.
After publishing and acknowledgement of the works of M. Scriven (Scriven, 1980), K. Weiss (Weiss, 1972), and M. Patton (Patton, 1972), the theoretical base of the assessment of budget programs began to be established (the end of the 1980s of the 20th century). Later, the work of G. Hatry (Hatry, 2005) was published that was aimed at the monitoring of the effectiveness of the managerial activity.
It should be mentioned that S.I. Ozhegov understood the criterion as a measure of assessment (Ozhegov, 1988). S.L. Sorkin claims that regarding the efficiency, the characteristic or measure on the base of which its condition is determined, can solve, first of all, the qualitative part of its content, can describe in detail the main goal of the economic activity in the market conditions. At the same time the measure of efficiency shall express the qualitative determination (Sorokin, 2011).
In the opinion of V.I. Vybornov, a criterion is the main measure of verity and certainty of any phenomenon, it is the main feature that is the base of their assessment; it should be one (Vybornov, & Mavrischev, 1982).
In our opinion, it is possible to use a system of criteria because any system is diversified – commercial, nationwide, branch, etc. and the criteria can have various forms of expression. Criteria determine the informatory idea of the efficiency and the informatory form of the efficiency is expressed by the indexes. Furthermore, taking into account the specific character of activity of the object only particular factors among the mentioned ones can be assessed (Belyaev, 2012).
On the base of the abovesaid, in our opinion, when estimating the budgetary costs aimed at the implementation of budget programs, budget efficiency should be taken into account. Moreover, the result of budget programs is mainly budget efficiency and social efficiency.
P.L. Vilensky, V.N. Livshits, and S.A. Smolyak offer to understand budget efficiency as the "value of the excess of the achieved result over the budget costs" (Vilensky et al., 2001). Regarding the social efficiency, it reflects the social consequences of the implementation of a budget program for the society. A.V. Marakuev (Marakuev, 2001), E.B. Atkinson (Atkinson, & Stiglitz, 1980) understand the social effect as a "result of impact (reaction) of a program on the target group or the society on the whole".
Thus, when estimating budget programs, the main factor is the assessment of efficiency based upon the system of monitoring that is formed during the implementation of a budget program at its various stages. Moreover, the efficiency of a budget program is a budget efficiency the base of which is the ratio of the social effect to the budget costs.
Nowadays, the formation of the methodological approaches to increasing the efficiency of budget costs within the frameworks of implementation of budget programs is urgent.
To achieve the efficiency of budget costs, the Ministry of Finance of the Russian Federation together with the Audit Chamber approved the method of assessment of the efficiency of budget costs on June 30, 2014. It took into account the peculiarities of the particular types of costs. Besides the main goal, that is, the assessment of the efficiency of budget costs, this method has the aim to create the conditions for formation of the complex assessment of the efficiency of budget costs that could combine the separate directions of such assessment and provide a fundamental analysis (final, indirect and also direct results) of the budget funds disbursement. The assessment of budget costs within the framework of this method shall be performed during planning of the state (municipal) programs and non-program measures and at the stages of their execution. The assessment of budget costs shall be performed in complex regarding the interrelated directions of budget costs and their types for the implementation of state, regional and municipal programs and also non-program part and for the implementation of actions (Ministry of Finance of the Russian Federation, 2014). Basing upon the abovesaid, this method functions as the backbone document on the base of which the methods of assessment of the efficiency of budget costs should be formed.
M. Scriven offered the method of assessment of the efficiency of the budgetary costs aimed at the rendering of budgetary services. The base is a comparison of the calculation of the cost of service in the budgetary sector with the analogous services developed by the non-state sector (Scriven, 1980). As a disadvantage of the method we can note the lack of possibility to estimate the services that do not have analogues in the market.
M.Yu. Patton offered the method of analysis of expenditures and results. This method allows calculating the cost of one unit of result. The costs of result of this unit will be the index of efficiency (Patton, 1997). The practical complexity of this method is the absence of the norm of cost of the result of the unit; therefore, an average index of the entire set of analogous institutions or some other conditionally ideal variant will be the base for comparison. This method can be called the method of ratio of conditional efficiencies.
The method offered by D.Yu. Zavyalov gives the possibility to estimate the changes of needs on the base of the calculation of budgetary costs. The efficiency of budgetary costs is determined as a ratio of the change of needs satisfaction of the population in the budget service to the change of volume of budgetary costs for its rendering. To measure the degree of the needs satisfaction, it is required to perform sociological research (Zavyalov, 2008). The complexity of this method should be mentioned regarding the conducting of sociological research because the verity of the conclusion depends upon the quality of the conducting, that is, the expenditure of funds will be considered efficient in case of prevailing of the degree of satisfaction of the society over the budgetary costs.
The most widely used is, in the opinion of A.A. Klimov, the assessment of the efficiency of budgetary costs for the following directions: at the stage of implementation of the program (preliminary, intermediate, generalizing); according to the intended use of the assessed program (assessment of need, quality of the preparation of the program, effectiveness, efficiency, "cost-profit", productivity and impact) (Klimov, 2009).
M.S. Efremova offers the stage-by-stage assessment of the efficiency of budgetary costs that is the following step consequence: "1 – selection and calculation of the indexes of the effectiveness of the implemented budgetary policy by compiling of the selection table, 2 – calculation of an arithmetical average and quality standard of every index of the effectiveness of budgetary costs for the rendering of services and measures of social support; 3 – calculation and estimation of the modular qualitative deviations; 4 – conclusion of the level of the efficiency of the budgetary costs of the considered direction of financing. Method of assessment of the efficiency of budgetary costs is based upon the calculation of the index of the modular qualitative deviation from the quality standard of rendering of budgetary services. The degree of deviation from the standard will be the reason for reconsideration of this type of budgetary costs in the next planning period" (Efremova, 2011).
The main principle of the approach offered by S.I. Karandasov is making a rating of the federal target programs. The assessment of the efficiency of federal target programs is made according to such criteria as: "correspondence to the legally approved recommendations, mathematical apparatus should be developed, the indexes of assessment of the efficiency should be informative" (Karandasov, 2012). It should be mentioned that the composed ratings will allow obtaining the objective representation of the quality of the implementation of federal target programs. Moreover, the ratings of federal target programs will create the competitive atmosphere among the controllers of budgetary funds and this, in its turn, will promote them to the development of assessment methods.
The performed review of the methodological approaches to the assessment of the efficiency of budgetary costs for implementation of budget programs showed that there is no united methodical tool in modern studies. Nevertheless, the study of problems of the assessment of the efficiency of implementation of budget programs for the rational distribution of budget funds does not cause doubt and requires the formation of the methodological foundations of the said scientific problem.
In this regard, the comparative and analytical analysis of the existing methods of assessment of the efficiency of implementation of budget programs of the municipal institutions of the Russian Federation was performed. When choosing the number and list of municipalities for the analysis of the methods of assessment of the efficiency of implementation of budget programs, the authors of the present article followed the principle of necessity and sufficiency.
The comparison of the methodical principles of assessment of the efficiency of implementation of budget programs is offered to be performed according to the following criteria: availability of the calculation of the indexes of effectiveness and efficiency of the measures, effectiveness and efficiency of a budget program, coefficients of significance (weight) of the indexes (values, criteria), general efficiency of a budget program (sub-programs, actions), degree of achievement of the set social and economic goals and also the satisfaction of the target groups of consumers of the state (municipal) services, availability of the expert estimation of the efficiency, description of risks of implementation of a budget program and mechanisms of their control.
The comparative analysis of the methodological principles of assessment of efficiency of implementation of budget programs of the municipalities is shown in Table 2.
Table 2. The comparative analysis of the methodological principles
of assessment of the efficiency of implementation of budget programs
of the municipalities
Municipality |
Efficiency assessment of implementation of municipal programs |
Voronezh (Order of Assessment of the Efficiency of Implementation of the Municipal Programs of the Voronezh City District, 2014) |
|
Kaliningrad (On Approval of the Order of Development, Implementation and Assessment of the Efficiency of the Municipal Programs of the Kaliningrad City District, 2014) |
|
Obninsk (On Approval of the Order of Decision-Making of the Development of the Municipal Programs of the Municipality "Obninsk", Their Formation and Implementation and the Order of Assessment of the Efficiency of Implementation of the Municipal Programs of Municipality "Obninsk, 2013) |
|
Ukhta (On Approval of Methodological Instructions of Development and Implementation of the Municipal Programs MICD Ukhta, 2013) |
|
Orsk (Order of Control and Assessment of the Efficiency of Implementation of the Municipal Programs, 2013) |
|
Tomsk (On Approval of the Order of Assessment of the Efficiency of Implementation of the Municipal Programs of the Municipality "Tomsk", 2015) |
It includes two stages: the assessment of the efficiency of implementation of subprograms of the municipal program and the efficiency of implementation of the municipal program on the whole. The assessment of the efficiency takes into account the degree of implementation of actions of a subprogram (variants of calculation depending upon the dynamics of the actual values), coefficient of the degree of achievement of goals and solution of the problems of subprogram, coefficient of efficiency of implementation of subprograms. The rating for the qualitative assessment of the efficiency of implementation of subprograms is presented. The assessment of the efficiency of implementation of the municipal program on the whole is estimated through the sum of the following coefficients: efficiency of implementation of the subprograms, achievement of the indexes of the goals of the municipal program, quality of control of the municipal program, efficiency of implementation of the providing subprogram taking into account their weight values. The rating is provided for the qualitative assessment of the efficiency of implementation of the municipal program. Also, the assessment of the efficiency of implementation of the municipal program is performed that does not include the subprograms regarding which the formulas of calculation are provided. The score and weight system of indexes assessment is provided. |
Cheboksary (On Approval of the Method of Assessment of the Efficiency of the Municipal Programs of Cheboksary, 2015) |
It provides the assessment of the efficiency of municipal programs on the base of the complex and integrated criteria. The complex criteria reflect the main characteristics of the municipal program. The integrated criterion takes into account the values of the complex criteria corrected to the weight coefficient. The score and weight system of assessment of the complex criteria is provided. The method takes into account the following correspondence: the goals of the municipal program to the priority of the policy of the local government authorities of Cheboksary during implementation of the municipal program, goals of the subprograms of the municipal program to the tasks of the municipal program, actually achieved and planned values of the target indexes and indexes of the municipal program (subprograms), implementation of the main actions of the subprogram to the target indexes and values of the municipal program (subprograms), budget funds disbursement, planned values of financing of the municipal program at the expense of the budget funds of Cheboksary to the sums of financing approved by the resolution of the budget of the city, planned and fulfilled main actions of subprograms, planned and actual terms of putting into operation of the capital construction projects provided by the municipal program, actual investment expenses to the volumes of the budget funds of Cheboksary provided by the municipal program. The correcting of the target indexes and values of the municipal program (subprograms) is provided when making changes, interdepartmental interaction for implementation of the municipal program. The breaches of the financial discipline are revealed and also the inefficient and improper use of the budget funds for implementation of the municipal program (subprograms) revealed during the inspection and review measures performed by the authorities of the external and internal state (municipal) financial control. |
Naryan-Mar (On Approval of the Order of Development, Implementation and Assessment of the Efficiency of the Municipal Programs of the Municipality "City District Naryan-Mar", 2013) |
It includes the following directions for assessment: the correspondence of the program to the priorities of the social and economic development, level of the target indexes of the municipal program, level of financial provision of the municipal program, level of fulfilment of program measures, organization of the management and control of the execution of the municipal program. The method does not contain the formulas for calculation of the criteria. Every criterion of the program efficiency is determined according to the score system of assessment in conformity with the percent gradation or by means of objective assessment depending upon the content of criterion. |
Thus, when analyzing the methodological instructions of the assessment of the efficiency of implementation of municipal programs, the following was revealed:
– most methodological instructions do not contain the formulas of calculation and information of the assessment criteria until the moment of execution. The assignment of weight to values (indexes, criteria) of assessment seems unreasoned. There is no description of risks of implementation of a budget program and mechanism of its control.
– the fuzzy representation of the indexes of efficiency and effectiveness is observed. Moreover, on the base of dividing of the actual values of the indexes into the planned values we cannot make a conclusion about the efficiency of implementation of the budget program. On the base of the similar calculation, we can conclude, most likely, on the result of some program action but not about its efficiency. The necessity of the development of indexes of the efficiency and effectiveness for the particular actions is provided by the fact that as a result there could be the possibility of making more qualitative managerial decisions about the correction of tasks, volumes of finances, correction of terms, etc.
– in most methodological instructions it is recognized that if the fact of improper use of budget funds is absent, the application of budget costs is considered efficient. Moreover, at the desired dynamics of the planned activities, the conclusion is made about the efficiency of implementing of budget programs.
– values (indexes, criteria) of assessment are developed by every municipal institution independently and this makes the comparative analysis of the efficiency of implementation of the municipal programs more difficult.
Summarizing the main provisions of the research, we will conclude that regulatory acts of the municipalities establishing the procedure of assessing the efficiency of implementation of budget programs require the adjustment or the critical changes of the above-mentioned positions.
Atkinson, A., & Stiglitz, J. (1980). Lectures on Public Economics. NY: McGraw-Hill.
Belyaev, M.I. (2006). Effektivnost [Efficiency]. Retrieved June 11, 2017, from http://www.milogiya2007.ru/micro27.htm.
Dolan, E.G., & Domnenko, B.I. (1994). Ekonomiks: anglo-russkii slovar-spravochnik [Economics: English-Russian Dictionary-Handbook] (Trans. from English). Moscow: Lazur. (p. 544).
Efremova, M.S. (2011). Povyshenie effektivnosti byudzhetnykh raskhodov v usloviyakh byudzhetirovaniya, orientirovannogo na rezultat [Improvement of Efficiency of Budget Costs in Conditions of Result-Oriented Budgeting]. Volgograd State Universit, Volgograd. (p. 26).
Hatry, G.P. (2005). Monitoring rezultativnosti v obshchestvennom sektore [Monitoring of Effectiveness in the Public Sector]. Moscow: IEK.
Karandasov, S.I. (2012). Perspektivy primeneniya programmno-tselevogo printsipa organizatsii raskhodov federalnogo byudzheta v Rossiiskoi Federatsii [Perspective of Use of the Program-Goal Principle of Costs of Federal Budget in the Russian Federation]. Vestnik Moskovskogo gosudarstvennogo oblastnogo universiteta, 1, 35-41.
Klimov, A.A. (2009). Povyshenie effektivnosti byudzhetnykh raskhodov [Improvement of Efficiency of Budget Costs]. Moscow: Izdatelstvo "Delo" AKhN. (p. 520).
Kovalev, V.V. (2007). Finansovyi menedzhment: teoriya i praktika [Financial Management: Theory and Practice] (2nd ed., revised and enlarged). Moscow: KC Welby, Prospect. (p. 1024).
Marakuev, A.V. (Ed.). (2001). Integrirovannaya informatsionnaya sistema sotsialno-ekonomicheskikh pokazatelei [Integrated Informational System of Social and Economic Indicators]. Moscow: CEMI RAS.
Mazur, I.I., Shapiro, V.D., & Olderroge, N.G. (2003). Effektivnyi menedzhment [Efficient Management]. Moscow: Vysshaya shkola. (p. 555).
Mescon, M.H., Albert, M., & Hedouri, F. (2007). Osnovy menedzhmenta [Essentials of Management] (O.I. Medved, Trans., 3rd ed.). Moscow: Williams. (p. 664).
Ministry of Finance of the Russian Federation. (2014). Kriterii i metodika otsenki effektivnosti byudzhetnykh raskhodov s uchetom osobennostei, opredelennykh vidom raskhodov [Criteria and Methods of Assessment of the Efficiency of Budget Costs Taking into Account the Peculiarities Determined by the Type of Costs]. Retrieved June 11, 2017, from gov.cap.ru›…108…metodika_ocenki_effektivnosti.docx
Ob utverzhdenii metodicheskikh ukazanii po razrabotke i realizatsii munitsipalnykh programm MOGO "Ukhta" [On Approval of Methodological Instructions of Development and Implementation of the Municipal Programs MICD Ukhta]. (2013, September 4). Resolution of the Administration of MICD Ukhta No. 1634. Retrieved June 11, 2017, from http://komi7.org/2013/11/07/k11342.htm
Ob utverzhdenii metodiki otsenki effektivnosti munitsipalnykh programm goroda Cheboksary [On Approval of the Method of Assessment of the Efficiency of the Municipal Programs of Cheboksary]. (2015, December 30). Resolution No. 3827. Retrieved June 11, 2017, from http://gov.cap.ru/SiteMap.aspx?gov_id=81&id=1730177
Ob utverzhdenii poryadka prinyatiya resheniya o razrabotke munitsipalnykh programm munitsipalnogo obrazovaniya "Gorod Obninsk", ikh formirovaniya i realizatsii i poryadka provedeniya otsenki effektivnosti realizatsii munitsipalnykh programm munitsipalnogo obrazovaniya "Gorod Obninsk" [On Approval of the Order of Decision-Making of the Development of the Municipal Programs of the Municipality "Obninsk", Their Formation and Implementation and the Order of Assessment of the Efficiency of Implementation of the Municipal Programs of Municipality "Obninsk"]. (2013, October 16). Resolution No. 1869-P. Retrieved June 11, 2017, from http://docs.cntd.ru/document/466200395
Ob utverzhdenii Poryadka provedeniya otsenki effektivnosti realizatsii munitsipalnykh programm munitsipalnogo obrazovaniya "Gorod Tomsk" (s izmeneniyami na 30 dekabrya 2016 goda) [On Approval of the Order of Assessment of the Efficiency of Implementation of the Municipal Programs of the Municipality "Tomsk" (changes valid on December 30, 2016)]. (2015, December 30). Resolution No. 1348. Retrieved June 11, 2017, from http://base.garant.ru/7782420/
Ob utverzhdenii Poryadka razrabotki, realizatsii i otsenki effektivnosti munitsipalnykh programm gorodskogo okruga g. Kaliningrada [On Approval of the Order of Development, Implementation and Assessment of the Efficiency of the Municipal Programs of the Kaliningrad City District]. (2014, July 8). Resolution No. 1040. Retrieved June 11, 2017, from http://www.klgd.ru/economy/programs
Ob utverzhdenii Poryadka razrabotki, realizatsii i otsenki effektivnosti munitsipalnykh programm munitsipalnogo obrazovaniya "Gorodskoi okrug "gorod Naryan-Mar"(s izmeneniyami na 29.04.2016) [On Approval of the Order of Development, Implementation and Assessment of the Efficiency of the Municipal Programs of the Municipality "City District Naryan-Mar" (changes valid on April 29, 2016)]. (2013, July 30). Resolution No. 1496. Retrieved June 11, 2017, from http://docs.cntd.ru/document/411713930
Ozhegov, S.I. (1988). Slovar russkogo yazyka [Dictionary of the Russian Language] (14th ed.). Moscow: Russkii yazyk. (p. 815).
Patton, M.Q. (1997). Utilization-Focused Evaluation. Thousand Oaks, CA: Sage Publications.
Patton, M.Q. (1997). Utilization-Focused Evaluation. Thousand Oaks, CA: Sage Publications. Retrieved June 11, 2017, from http://www.msses.ru/lib/fund/gifts/newbooks2005.php
Perevalov, Yu.V., Gimadi, I.E., & Dobrodey, V.V. (1997). Effektivnost deyatelnosti privatizirovannykh predpriyatii i otraslei: metodicheskie voprosy analiza [Efficiency of Activity of the Privatized Enterprises and Branches: Methodical Questions of Analysis]. Obshchestvo i ekonomika, 9-10, 23-26.
Poryadok kontrolya i provedeniya otsenki effektivnosti realizatsii munitsipalnykh programm [Order of Control and Assessment of the Efficiency of Implementation of the Municipal Programs]. (2013, August 20]. Resolution No. 6082-p. Retrieved June 11, 2017, from orsk-adm.ru›sites/default/files/podprogramma
Poryadok provedeniya otsenki effektivnosti realizatsii munitsipalnykh programm gorodskogo okruga gorod Voronezh [Order of Assessment of the Efficiency of Implementation of the Municipal Programs of the Voronezh City District]. (2014, November 7). Resolution No. 1580. Retrieved June 11, 2017, from http://www.voronezh-city.ru/administration/structure/detail/10765
Scriven, M. (1980). The Logic of Evaluation. Inverness, CA: Edgepress.
Scriven, M. (1980). The Logic of Evaluation. Inverness, CA: Edgepress. Retrieved June 11, 2017, from http://iakemachina.com/OSSA/pdf/85Q Scriven.pdf
Sorkin, S.L. (2011). Effektivnost vneshneekonomicheskoi deyatelnosti: ponyatie, izmerenie i otsenka: monografiya [Efficiency of Foreign Economic Activity: Notion, Measurement and Estimation: Monography]. Grodno: GSU. (p. 130).
Vilensky, P.L., Livshits, V.N., & Smolyak, S.A. (2001). Otsenka effektivnosti investitsionnykh proektov: Teoriya i praktika [Assessment of Efficiency of Investment Projects: Theory and Practice]. Moscow: Delo.
Vybornov, V.I., & Mavrischev, V.S. (1982). Ekonomicheskaya effektivnost promyshlennogo proizvodstva: ucheb. posobie dlya vuzov [Economic Efficiency of Industrial Production: Textbook for Higher Educational Institutions]. Minsk: Vysshaya shkola. (p. 270).
Weiss, C. (1972). Evaluation Research: Methods for Assessing Program Effectiveness. Englewood Cliffs, NJ: Prentice-Hall Press.
Zavyalov, D.Yu. (2008). Otsenka effektivnosti byudzhetnykh raskhodov: sravnitelnyi analiz [Assessment of Efficiency of Budget Costs: Comparative Analysis]. Finansy, 10, 6-10.
1. Tomsk State University of Architecture and Building, 634003, Russia, Tomsk, Solyanaya Sq., 2. E-mail: ufimtseva80@mail.ru
2. Tomsk State University of Architecture and Building, 634003, Russia, Tomsk, Solyanaya Sq.,
3. Tomsk State University of Architecture and Building, 634003, Russia, Tomsk, Solyanaya Sq.,
4. Tomsk State University of Architecture and Building, 634003, Russia, Tomsk, Solyanaya Sq.,
5. Tomsk State University of Architecture and Building, 634003, Russia, Tomsk, Solyanaya Sq.,
