Hi, The Virtualizor Team has released Virtualizor 2.9.6 In this version we have added some revolutionary features like Cloud Billing system, CEPH storage and many more. Here is the change list : 1) [Feature] Virtualizor will now have the capability to do all invoicing via our new Cloud Billing feature. A user can register an account and add funds into his / her account. The user can then create VMs and delete them at his will. We have also added HOURLY BILLING support and all this can be done from the Virtualizor Panel itself. Please refer to the following guide for setting up Cloud billing : http://virtualizor.com/wiki/Virtualizor_Cloud_Billing_System NOTE: Cloud Billing is in public Beta. Also, the enduser experience for cloud billing has been documented : https://virtualizor.com/wiki/Cloud_Billing_Enduser_Experience 2) [Feature] Added CEPH storage support. Here is the guide to setup CEPH cluster and use it in Virtualizor : https://www.virtualizor.com/wiki/Add_New_Storage#Create_Storage_3 3) [Feature] Added Migrate Master utility so that you can elect any slave as a new Master OR shift the Master Only Node to a new Node : https://virtualizor.com/wiki/Migrate_Master_New 4) [Feature] Added support for migration of VMs from OpenVZ 6 to OpenVZ 7. You can now migrate your old OpenVZ 6 to the new OpenVZ 7 platform. 5) [Feature] Now while rebuilding the VM or Reinstalling the OS from the Enduser panel, only primary disk will be formatted if "Format Primary disk" is checked. By default it will format all the disks of the VM. 6) [Feature] Added integrity check for Remote Backups. This will ensure that the backup was completed successfully and correctly. 7) [Feature] Now Virtualizor supports Backup / Restore of VMs with Multiple disk(s). Previously it was taking backup of Primary disk only. 8) [Feature] Virtio disk driver option added for Proxmox KVM. This gives better disk I/O for VMs. 9) [Feature] Template creation added for Openvz 7 VMs and CTs. 10) [Feature] Backup features added for OpenVZ 7 VMs and CTs. 11) [Feature] Now TC rules on OpenVZ VMs will be applied on live VM. 12) [Feature] Admins can set the email notification threshold for bandwidth usage of VMs as well. However, the enduser threshold will be given first preference. 13) [Feature] Users can now specify a email notification threshold for bandwidth usage of their VMs. By default it will be 80% for all VMs. 14) [Feature] Added setting in Admin Panel to disable VGA option in VPS configuration wizard in Enduser Panel. It can be disabled from Admin Panel -> Master setting -> Enduser Settings section -> Disable VGA option 15) [Feature] Added IPv6 network throttling for all virtualizations. Speed capping will be applied to IPv6 as well. NOTE : On Current running VMs / CTs Admin need to execute the following command manually on each server to avoid issues. root# /etc/init.d/zzvirtservice restart Above command will remove the old TC rules and rewrite the rules for existing VMs / CTs on the server. 16) [Feature] Secondary ISO support added for OpenVZ 7 KVM VMs. 17) [Feature] Added option to add a NOTE for an IP Address. Admin can add a note for any IP Addresses from the Admin Panel -> List IPs -> Edit any IP -> There you will find the field for adding a note for that IP. 18) [Feature] Added Timezone selection option for VMs from Enduser Panel -> VPS configuration -> Select VPS Timezone. This will require stop and start of the VPS. 19) [Feature] Now IP Pools can be selected while making VPS Plans. This will enable Admins to select particular IPs for particular regions. 20) [Task] Now IPv6 subnet(s) will have correct netmask prefix and routing options if a VPS has only IPv6 subnet(s) assigned to it. 21) [Task] Cron emails sent to the Admin will have the server Hostname with IP address in any email wherever applicable. 22) [Task] Updated NoVNC client library to latest version. This brings in a better user experience. 23) [Task] Add Storage wizard improved to show information related to the storage type chosen. 24) [Task] Now OpenVZ KVM VM will be created with virtio disk drivers. Before it was creating with scsi-virtio and due to that windows ISOs were not getting loaded. 25) [Task] KVM VMs with internal IPs will have MAC address assigned to internal IPs as well. Before it was not assigning the MAC address to internal IPs. 26) [Task] From version 2.9.6 if VM fails to suspend, Virtualizor will power off the VMs. 27) [Task] Added Storage Overcommit handler for OLD WHMCS module. This was causing the issue for those users who are still using OLD WHMCS (V1) Module. For this users need to update the WHMCS module. 28) [Task] Added utility to disable Two Factor Authentication for ROOT. Here is the guide to the same : https://virtualizor.com/wiki/Enable/Disable_Two_Factor_Authentication#Disable_for_root 29) [Task] Added Speed Cap column in VPS statistics wizard. This will show that the VM(s) has speed capping or not. 30) [Task] Allowed email addresses to start with an underscore '_'. 31) [Task] Enduser ListVS - removed VPS Name as it used to create confusion. 32) [Bugfix] OpenVZ 6 Containers and OpenVZ 7 KVM VMs Hostname issue has been fixed. 33) [Bugfix] While editing the VMs from Manage VPS wizard, internal IPs were getting removed (if any). This is fixed. 34) [Bugfix] Rescue mode Enable/Disable was not working from API calls. This is fixed. 35) [Bugfix] OpenVZ 7 CT creation was giving error as "OS template not found". This is fixed now. 36) [Bugfix] Proxmox was not working if cluster setup was made on internal network. Now API calls will be made on 127.0.0.1 thus resolving the issue. 37) [Bugfix] On Xen, network of any random VPS was not working while updating network rules for one VPS. This is fixed. 38) [Bugfix] Ebtable rules were not getting removed on some servers creating duplicate rules. This is fixed. 39) [Bugfix] On Centos 7+ Host Nodes, while unsuspending the VM which is already unsuspended was causing restart of the VM. This is fixed. 40) [Bugfix] After reinstalling the OS, MAC of the VM was getting changed and causing network issues. Now MAC will not be changed after reinstalling OS. 41) [Bugfix] On XEN CPU% parameter for guest was not set due to wrong config name in configuration file. This is fixed. 42) [Bugfix] On some XenServers, finding guest_tool.iso was taking some time due to which VM provisioning was also taking time. This is fixed. 43) [Bugfix] On OpenVZ 7, autostart setting (on node reboot) for CT was not working. This is fixed. 44) [Bugfix] On OpenVZ 7 if OS Template was not there from which the VM is created, Virtualizor was giving error of OS template not found while editing the VM. This is fixed. 45) [Bugfix] On Ubuntu 16.04 host, OS Services wizard in Admin panel was not working. This is fixed. 46) [Bugfix] If Windows VM had different username (i.e. Other than Administrator) and created VM was on Slave server, at that time Virtualizor was not able to change the password for that VM. This is fixed. 47) [Bugfix] On Amdin panel, Forgot Password email template was showing blank fields. This is fixed. 48) [Bugfix] From enduser panel, while reinstalling the OS, password was not getting set for special characters in password. This is fixed. 49) [Bugfix] ACL option for Enduser ISO was not there in ACL wizard in Admin Panel. This is fixed. 50) [Bugfix] HA Proxy rules were not getting applied for the VMs which were created on Slave servers. This is fixed. 51) [Bugfix] HA Proxy rules were not working if applied for a domain when a subdomain of the same domain was added. The requests were forwarded to the IP of the subdomain. This is fixed. 52) [Bugfix] Proxmox LXC containers with file storage were not getting started due to wrong disk name. This is fixed. 53) [Bugfix] IPv6 was not working for OpenVZ 7 VMs IF speed capping was enabled. This is fixed. ======================= Patch 1 ======================= 54) [Bugfix] Bandwidth Threshold emails were sent every 8 minute via cron to enduser(s) whose bandwidth exceeded the assigned limit. This is fixed and the email will be sent only once a month. 55) [Bugfix] Create VPS wizard was showing blank page if the current selected server did not have IP Pool. This is fixed. This issue was introduced in Virtualizor 2.9.6.0. 56) [Bugfix] While making any changes in VPS configuration option from WHMCS module, it was not updated in Virtualizor panel. This is fixed. This issue was introduced in Virtualizor 2.9.6.0. ======================= Patch 2 ======================= 57) [Bugfix] If VPS template was virtio enabled and while VPS creation swap is created and virtio was disabled, Virtualizor was not handling it properly. This is fixed. 58) [Bugfix] In Proxmox KVM VLAN Bridge was not getting added to config even if it was defined in IP pool. This is fixed. 59) [Bugfix] In OpenVZ while restore to new VPS action was performed, Virtualizor was not cleaning up the unwanted files. This is fixed. 60) [Bugfix] On the Slave Settings form for proxmox servers if the form was submitted with empty root password it was saved as empty causing the API calls to fail. This is fixed. 61) [Bugfix] On Admin dashboard if there are more servers added then extra space was displayed in "Server Statistics" section. This issue was introduce in 2.9.6.0. This is fixed. ======================= Patch 3 ======================= 62) [Feature] Added reCaptcha on Cloud user registration page. To enable captcha Admin needs to generate Secret key and site key as per the following guide : https://www.google.com/recaptcha 63) [Bugfix] While importing a virtio enabled VM from SolusVM, Virtualizor was converting the disk driver as IDE. This is fixed. 64) [Bugfix] While importing VMs from Proxmox with ZFS storage configured, Virtualizor was adding those VMs as file storage instead of ZFS. This is fixed. 65) [Bugfix] While creating template using winodws.php (CLI utility), if the Windows VM is created on Thin Pool, at that time Virtualizor was creating Temporary LV on VG instead of Thin-pool. This is fixed. NOTE: Thin Pool must be a Primary Storage for this to work. 66) [Bugfix] Launch VPS VNC wizard on Admin Panel was dying, if there were more than 3000 VMs. This is fixed. 67) [Bugfix] In Cloud panel, the Balance was not getting added accurately in account. This is fixed. 68) [Bugfix] VM creation was failing on slave server if "Delete OS Template on Slave Server" was enabled. This is fixed. 69) [Bugfix] On 32 bit Host servers, VPS backups were failing silently without reporting the actual error. This is fixed. 70) [Bugfix] Added URL check for Add ISO wizard on Admin Panel and Enduser Panel. This is fixed. Now only HTTP, HTTPS and FTP protocols are allowed. 71) [Bugfix] Billing Symbol was shown as $ (Dollar) even if it was changed to other currency from Admin Panel -> Billing wizard. This is fixed. 72) [Bugfix] Added ntfsfix checking for file base windows VMs at the time of resizing the file system. It was causing issue on some servers failing to boot the VM. This is fixed. 73) [Bugfix] In backup plan if the Backup Path was not absolute path, it was causing segfault and hence the backups failed. Now absolute path check has been added to avoid this issue. 74) [Bugfix] Email ending with "." (DOT) were getting accepted. This is fixed. 75) [Bugfix] While creating VMs from billing enabled Cloud panel, OS template was getting selected from VPS plan instead of Cloud user selection. This is fixed. 76) [Bugfix] Newly registered Cloud User was able to create VMs even though there were no sufficient funds in their account. This is fixed. 77) [Task] All users were getting synced to all Slave servers even if there were no VMs available for those users on slave servers. Now users will be synced to slave servers only if they have VMs on that slave server. This improves syncing time between slave servers. ======================= Patch 4 ======================= 78) [Bugfix] Billing module related fixes. Virtualizor supports OpenVZ, OpenVZ 7, Xen, XenServer, LXC and Linux-KVM on CentOS and Ubuntu and they all can be managed from the Master. We are adding more features and if you have any suggestions, do let us know. We hope you enjoy this version of Virtualizor. Regards, The Virtualizor Team