Utilizando concorrência nas consultas ao EPM Server

Vamos implementar o uso da biblioteca padrão do Python concurrent.futures para fazer consultas ao EPM Server, trazendo possíveis ganhos de peformance, principalmente quando se possui muitas variáveis a serem pesquisadas.

In [66]:
import epmwebapi as epm
import numpy as np
import datetime

##Método para não expor usuário e senha do EPM no código fonte
import getpass
user = input('EPM user:')
password = getpass.getpass("EPM password:")

#crie o objeto de conexão informando os endereços do EPM Webserver(Authentication Port e WEB API Port), usuário e senha.
connection = epm.EpmConnection('http://localhost:44333', 'http://localhost:44332', user, password)

#periodo da consulta 
end_date = datetime.datetime.now()
ini_date= end_date - datetime.timedelta(days=2)
queryPeriod = epm.QueryPeriod(ini_date,end_date)

#função para buscar dados
def get_basicvariable_rawvalues(epmconnection, bvname, period):
    
    bv = connection.getDataObjects([bvname])
    data = bv[bvname].historyReadRaw(period)
    print(bvname+' lenght:{}'.format(str(len(data))))
    return data

#20 basic variables que serão pesquisadas
bvname = ['ADM_Temperature','EPMDev_Temperature','E3Dev_Temperature', 
          'TI_Temperature', 'PowerDev_Temperature', 'TechnicalSupportTemperature',
         'Auditorio_Temperature','Treinamento_Temperature','MobileDev_Temperature',
         'MeetingRoom11th_Temperature', 'MeetingRoom10th_Temperature','MeetingRoom12th_Temperature',
         'TIServidores_Temperature', 'Dev12th_Temperature', 'Dev11th_Temperature', 
         'ADM_TemperatureSP', 'MeetingRoom10th_TemperatureSP', 'EPMDev_TemperatureSP', 
         'PowerDev_TemperatureSP', 'MeetingRoom11th_TemperatureSP']
EPM user:elipse\kotres
EPM password:········

Executando a consulta sequencialmente em um laço

Verificamos que o retorno ocorre na ordem da lista bvname.

In [67]:
%%time

data = []

for bv in bvname:    
    data.append(get_basicvariable_rawvalues(connection, bv, queryPeriod))        
    
ADM_Temperature lenght:870
EPMDev_Temperature lenght:811
E3Dev_Temperature lenght:146
TI_Temperature lenght:1491
PowerDev_Temperature lenght:251
TechnicalSupportTemperature lenght:111
Auditorio_Temperature lenght:2290
Treinamento_Temperature lenght:2498
MobileDev_Temperature lenght:2816
MeetingRoom11th_Temperature lenght:197
MeetingRoom10th_Temperature lenght:341
MeetingRoom12th_Temperature lenght:1637
TIServidores_Temperature lenght:3128
Dev12th_Temperature lenght:2241
Dev11th_Temperature lenght:111
ADM_TemperatureSP lenght:1
MeetingRoom10th_TemperatureSP lenght:1
EPMDev_TemperatureSP lenght:1
PowerDev_TemperatureSP lenght:7
MeetingRoom11th_TemperatureSP lenght:7
Wall time: 6.57 s

Executando a consulta com concorrência

  • Para esse caso houve redução de cerca de 57% no tempo total da consulta.
In [68]:
%%time

import concurrent.futures

#máximo de consultas paralelas
MAX_WORKERS = 10

data = []

#executa no máximo 10 consultas paralelas e no mínimo a quantidade de variáveis a serem consultadas (se forem menos de 10).
workers = min(MAX_WORKERS, len(bvname))

#usamos a clausula with para ter certeza que as threads serão fechadas logo após o termino da sua execução
with futures.ThreadPoolExecutor(workers) as executor:
    future_to_get_bv = {executor.submit(get_basicvariable_rawvalues, connection, bv, queryPeriod): bv for bv in bvname}    
    for future in concurrent.futures.as_completed(future_to_get_bv):
        data.append(future.result())
ADM_Temperature lenght:870
MeetingRoom11th_Temperature lenght:197
EPMDev_Temperature lenght:811
TechnicalSupportTemperature lenght:111
E3Dev_Temperature lenght:146
PowerDev_Temperature lenght:251
TI_Temperature lenght:1491
MeetingRoom10th_Temperature lenght:341Dev11th_Temperature lenght:111

ADM_TemperatureSP lenght:1
MeetingRoom10th_TemperatureSP lenght:1
Auditorio_Temperature lenght:2290
EPMDev_TemperatureSP lenght:1
PowerDev_TemperatureSP lenght:7
Treinamento_Temperature lenght:2498
MeetingRoom11th_TemperatureSP lenght:7
MobileDev_Temperature lenght:2816
MeetingRoom12th_Temperature lenght:1637
Dev12th_Temperature lenght:2241
TIServidores_Temperature lenght:3128
Wall time: 2.83 s