techware – trianglesis blog

Home » Category "techware"

WSGI mod, Python, Django, Celery and a nice fella virtualenv

By trianglesis 28.10.2019 28.10.2019 OS, programming, server, techware, Web

Here I’ll show how to use Python (3.8.0 in the current example, but legit for latest 3.+ versions, not for os.name == ‘NT’) and virtualenv to setup WGSI enabled website on Django (or any) with implemented Celery + Flower + RabbitMQ. This example is my own path used for automated test framework: Octopus (not available now, but have dev branch opened: Lobster) Here only configurations and setup for Python 3.8.0, virtualenv, WSGI, Django, Celery (worker and beat services), Flower and that’s it. Later I’ll show Django + Celery configuration when they run together (sort of). You may want to install Python at first: Read When your modern and fancy python is installed, we’ll make a virtual environment for our website.Short Read more…

Test automation or: How I Learned to Stop Worrying and Love Async init

By trianglesis 14.08.2018 16.08.2018 automation, programming

Morning everybody… I recommend reading this part of the blog only in the morning with a cup of dark and strong coffee. I’ll show you some new cases, and howto’s to help you automate routines in the developer’s life. There will be an example of my fails and win on different scenarios. The poor code I’ll include also. Very beginning Firstly – I’ll try to make this thread in English. And maybe later when I have time – to make Ukrainian and Russian translations, I’ll make them. “Octopus” story has started from a small plugin for Sublime Text editor. Later it will be ported to Atom. I’ve started developing “Octopus” on spring-summer 2016, and I’ve not finished it yet! New functionsShort Read more…

Python – Django Celery – кейс длиною в вечность.

By trianglesis 02.08.2018 02.08.2018 programming

Вкратце: когда добавляешь пачку тасок на Celery worker’ы, случается так, что парочка из них отваливается или засыпает или еще чего. Долго не доходили руки поправить это недоразумение, поэтому я просто рестартовал все воркеры раз в пару дней или прямо перед началом проверки рутины, в общем очень сложно было поймать и воспроизвести тот случай, когда без каких-либо ошибок воркер просто молчал. И наконец дошоли руки запилить кривофикс, время покажет насколько он толковый, суть в следующем: В сразу после момента определения свободного воркера – запускать app.control.heartbeat() и app.control.ping(timeout=10) и если ответ пришел (в виде [{‘alpha@host’: {‘ok’: ‘pong’}}]) – значит воркер готов и можно передавать ему таску, пачку тасков, рутину и тп. Если ответ пришел отличным от этого – даем ошибку и ничего не запускаемShort Read more…

Python 3 showing progressbar while subprocess.Popen

By trianglesis 23.09.2017 14.08.2018 programming, techware

Have an interesting case I want to share and save for future. One small problem in this system: https://github.com/trianglesis/BMC_TPL_IDE is showing when some process is running and was not hanged up. Sometimes we need to run a huge amount of checks and tests and would be great to know if something hangs before killing it mannually. I used different scenarios and mostly a module pregressbar2, and this is what I finally can compose: Make worker which will execute job sorf of randomly:


import time
import random

print("START SOMETHING")

for i in range(5):
    sec = random.randint(1, 3)
    print("Sleeping for: %d" % sec)
    time.sleep(sec)
    print("Making job, line: %d" % i)

print("END SOMETHING")

import time

import random

print("START SOMETHING")

for i in range(5):

sec = random.randint(1, 3)

print("Sleeping for: %d" % sec)

time.sleep(sec)

print("Making job, line: %d" % i)

print("END SOMETHING")

Make process executor:


import sys
import subprocess
import progressbar

# Draw spinner:
bar = progressbar.ProgressBar(max_value=progressbar.UnknownLength)

# Execute some job with multiple lines on stdout:
p = subprocess.Popen("C:\Python34\python.exe worker.py", shell=True, stdout=subprocess.PIPE,
                                                                     stderr=subprocess.PIPE)
# Lines will be collected in list:
result = []

# Until I get last line and the end of string:
while p.stdout is not None:

    # Update spinner on one step:
    # It will update only when any line was printed to stdout!
    bar.update()
    # Read each line:

    line = p.stdout.readline()
    # Add line in list and remove carriage return

    result.append(line.decode('UTF-8').rstrip('\r'))

    # When no lines appears:
    if not line:
        print("\n")
        p.stdout.flush()
        break

# Show finish message, it also useful because bar cannot start new line on console, why?
print("Finish:")
# Results as string:
print(''.join(result))

import sys

import subprocess

import progressbar

# Draw spinner:

bar = progressbar.ProgressBar(max_value=progressbar.UnknownLength)

# Execute some job with multiple lines on stdout:

p = subprocess.Popen("C:\Python34\python.exe worker.py", shell=True, stdout=subprocess.PIPE,

stderr=subprocess.PIPE)

# Lines will be collected in list:

result = []

# Until I get last line and the end of string:

while p.stdout is not None:

# Update spinner on one step:

# It will update only when any line was printed to stdout!

bar.update()

# Read each line:

line = p.stdout.readline()

# Add line in list and remove carriage return

result.append(line.decode('UTF-8').rstrip('\r'))

# When no lines appears:

if not line:

print("\n")

p.stdout.flush()

break

# Show finish message, it also useful because bar cannot start new line on console, why?

print("Finish:")

# Results as string:

print(''.join(result))

Voilla! But there is still a small problem – this bar will update it’s state only when process post something in stdout. Between posts it will stay calm for random amount of timeShort Read more…

VK_execute

By trianglesis 26.07.2017 26.07.2017 programming, techware

Для всех страждущих наконец выкладываю нормальные примеры использования VK Script: https://github.com/triaglesis/vk_execute Там все мои примеры и по работе и просто тестовые. Самый быстрый вот такого плана, без внутреннего парсинга. Как только добавляешь внутрь вк скрипта какой-то проверочный код – КПД резко падает. Мой совет – вк_скрипт только для сбора всего “сырого” – дальше обрабатывать уже на сереверной стороне.


var posts_cnt;
var posts;
var post_list = [];
var offset_posts = Args.global_offset_posts;
offset_posts = parseInt(offset_posts);
// var offset_posts = 0;
var run_count = 0;
var iter_count = 20;
while (iter_count > run_count )
    {posts = API.wall.get(
        {"owner_id":(Args.owner_id),
         "domain":(Args.domain),
         "offset":(offset_posts),
         "count":"100",
         "filter":"all",
         "extended":"0",
         "v":"5.42"}
        );
    post_list.push(posts.items);
    offset_posts = offset_posts + 100;
    run_count = run_count + 1;
    posts_cnt = posts.count;
    };
return {"posts_cnt": posts_cnt,
        "run_count":run_count,
        "iter_count":iter_count,
        "offset_posts": offset_posts,
        "posts": post_list};

var posts_cnt;

var posts;

var post_list = [];

var offset_posts = Args.global_offset_posts;

offset_posts = parseInt(offset_posts);

// var offset_posts = 0;

var run_count = 0;

var iter_count = 20;

while (iter_count > run_count )

{posts = API.wall.get(

{"owner_id":(Args.owner_id),

"domain":(Args.domain),

"offset":(offset_posts),

"count":"100",

"filter":"all",

"extended":"0",

"v":"5.42"}

);

post_list.push(posts.items);

offset_posts = offset_posts + 100;

run_count = run_count + 1;

posts_cnt = posts.count;

};

return {"posts_cnt": posts_cnt,

"run_count":run_count,

"iter_count":iter_count,

"offset_posts": offset_posts,

"posts": post_list};

Django, Apache2, Sublime Text 3 build system and developing process.

By trianglesis 04.05.2017 04.05.2017 programming, server, techware

At first be sure that you are using this path to run pure apache: http://trianglesis.org.ua/django-python-3-4-wamp-apache-2-4-23-pycharm This topic for those who wants to develop ON THE FLY with python and django and do not want to manually restart dev server each time when code changes AND you don’t like built in python web-server and want to run your dev site as it should run on apache. Sublime Text 3 I use it for fast code changing and writting because PyCharm can’t work fine with multiline carrets, for example. Also PyCharm is too heavy when you want just copy&paste some features and see how it work on web. So my PyCharm will be used for debug purposes. I hope you know what isShort Read more…

Django + Python 3.4 + WAMP Apache 2.4.23 + Pycharm

By trianglesis 27.04.2017 27.04.2017 programming, techware

Finally worked instance for hosting DEV site code on python and ability to resolve is as “on live” web server just on my laptop. A lot of different configurations changed but this is one working for me, and maybe can help you also. Better use EVERYTHING 32 bit for Window, you will suffer if you try to find some needed wheels for python 64bit on Windows 7 64bit. WAMP Apache 2.4.23 Download: wampserver3.0.6_x86_apache2.4.23_mysql5.7.14_php5.6.25-7.0.10 Here


NOTE: Install all needed C++ Redistr. packages which are listed in WAMP install guide!

NOTE: Install all needed C++ Redistr. packages which are listed in WAMP install guide!

Download mod_wsgi.so Here or Here (for everythin 32 bit and python 3.4 on Win7 you need: mod_wsgi-py34-VC10.so)


NOTE: Use mod_wsgi version which corresponds your python version! (Above in links you can find more info, just read carefully)

NOTE: Use mod_wsgi version which corresponds your python version! (Above in links you can find more info, just read carefully)

After WAMP install – you don’t need to run full programm (I just did not see any advantages of this and disable wampmysql service) All you need is here: C:\wamp\bin\apache\apache2.4.23 Place mod_wsgi-py34-VC10.soShort Read more…

Gjango, Pycharm – static files Windows

By trianglesis 26.04.2017 26.04.2017 programming, techware

This topic for those who broke his mind trying to understand how to make Pycharm run Django with static files I’ve made it, but be carefull it can break your mind too! The settings I use the most portable way to configure settings – so I can move it into centos web server without paths changes Win\Unix (I HOPE). There are common DIRs


BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# BASE DIR: D:\Projects\CODE\WEB\sites\www\smm_tools\smm_site

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

# BASE DIR: D:\Projects\CODE\WEB\sites\www\smm_tools\smm_site


PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))
# PROJECT_ROOT:     D:\Projects\CODE\WEB\sites\www\smm_tools\smm_site\main_code

PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))

# PROJECT_ROOT: D:\Projects\CODE\WEB\sites\www\smm_tools\smm_site\main_code


STATIC_ROOT = os.path.join(BASE_DIR, 'static')
#Static root:      D:\Projects\CODE\WEB\sites\www\smm_tools\smm_site\static

STATIC_ROOT = os.path.join(BASE_DIR, 'static')

#Static root: D:\Projects\CODE\WEB\sites\www\smm_tools\smm_site\static


STATIC_URL = '/static/'

STATIC_URL = '/static/'

Looks like ot works, but in really weird way: static files are working fine web page as template is loading some scc and js are loading and some are not Working css: In works in different ways – this is one of them which I think correct. In HTML code:


<link href="{% static 'css/bootstrap.css' %}" rel="stylesheet" type="text/css">
<link href="{% static 'css/custom.css' %}" rel="stylesheet" type="text/css">

In WEB: Working js:Short Read more…

Windows mirrored volume and hard reset

By trianglesis 20.08.2016 20.08.2016 hardware, OS, techware

When you doing a hardware reset for Windows Server 2012 Hyper-V with RAID1 you may catch “Volume Sync” For me to remember: http://serverfault.com/questions/107636/how-to-view-status-of-software-raid-1-resynching Read more…

No comments

Пост с именинниками Вконтакте

By trianglesis 28.07.2016 28.07.2016 programming, techware

Публикация поста с именинниками Вконтакте: Старый скриптец, который я использовал около года назад для выборки именинников и публикации поста с ними на стену сообществ Вконтакте. Выкладываю в публичный доступ, так как морально он устарел и код в нем выглядит ужасно, но подходит чисто для образовательных целей и “авось кому пригодится” мне бы он помог на первых порах изучения, может быть … Сам код можете смотреть здесь: В ближайшие дни, недели, месяцы я буду выкладывать все эти прошлые поделки, по мере того, как полностью перепишу их на новый манер, сейчас у меня основная задача – полностью унифицировать эти отрывки кода и сделать из них полноценную систему управления и автоматизации, поэтому, эти вот отрывки сами по себе уже никакой ценности не несутShort Read more…

Category: techware

WSGI mod, Python, Django, Celery and a nice fella virtualenv

Like this:

Test automation or: How I Learned to Stop Worrying and Love Async init

Like this:

Python – Django Celery – кейс длиною в вечность.

Like this:

Python 3 showing progressbar while subprocess.Popen

Like this:

VK_execute

Like this:

Django, Apache2, Sublime Text 3 build system and developing process.

Like this:

Django + Python 3.4 + WAMP Apache 2.4.23 + Pycharm

Like this:

Gjango, Pycharm – static files Windows

Like this:

Windows mirrored volume and hard reset

Like this:

Пост с именинниками Вконтакте

Like this:

Search

Автор

Recent Posts

Поделиться:

Like this:

Поделиться:

Like this:

Поделиться:

Like this:

Поделиться:

Like this:

Поделиться:

Like this:

Поделиться:

Like this:

Поделиться:

Like this:

Поделиться:

Like this:

Поделиться:

Like this:

Поделиться:

Like this:

Search

Автор

Recent Posts

Tags