Some of these complaints I would agree with - in particularly, we're not (yet) caching build resources - though we're working on this now. But most of these complaints I was quite surprised to hear; not an experience I would want someone to have or what I see from the majority of our customers. So this is certainly a place where I'll follow up with the author to learn more.
It's the entire experience. The UI is terribly clunky everywhere. The worst for me are pull requests. Incredibly tough to work with people on a pull request. I can't even point you to "a" particular problem - for us it's broken everywhere. From the weird file comparison with overlays to the missing support for (us) important file formats like ipython notebooks. The way notifications bubble up in a facebook like format on a pull request. The emails that have 20000 lines and never the exact information I need. The amount of clicks to go from comparing two files to open the current version. Other people mentioned the lacking caching support for builds, so I am not going to go into more detail here. Another missing piece for me: Where are my statistics. I'd love to see commit, PR, response, comment etc statistics per team member / teams / groups.
Using Azure DevOps board is a nightmare of its own...
btw: most of these issues are part of the whole "azure" experience. The Azure portal suffers from the exact same interface issues of horizontally stacking fields. One wrong click and your last edits have disappeared into the ether
But the Azure UI is so bad that people are willing to forego stuff like that just to avoid being on Azure.
The Azure product teams have asked for feedback from startup multiple times with consistent answers on UI - I'm no longer willing to believe that this isn't the consequence of some global UI mandate to make things look "windows-y"
I have managed to forget the search terms I entered, but the first thing I have stumbled on, is this :
https://www.w3.org/TR/WCAG20-TECHS/SL15.html
Keyboard shortcuts across Silverlight applications.
I have no idea what became of Silverlight. But I liked it very much more when I understood Silverlight was the future.
(how anyone understood the tech road maps at the time Silverlight was introduced, I can't imagine)
I personally see the problem is that search engines are disinterested in the furtherance of any kind of consistency capable of reducing the utility of a layer of discovery that they can provide. If you explained to your senator, that whoever dominates the browser market, effectively controls how all the information contained in the www is displayed and arranged by publishers, and the standards bodies intent to introduce semantic markup and accessibility across the www potentially can eliminate the need for search engines for a variety of constituencies and professional workers, no less cutting spam most certainly in the process, maybe they'd see the inherent conflict to be as undesirable as I see it to be.
I keep thinking I need to at least spend a while considering how possibly a browser might technically introduce undo / redo for arbitrary websites. This is what I would be trying to accomplish, if I held the necessary influence with the Edge team. Even the ability to be informed whatever it is that I just did to the page, would be enormously valuable to me and everyone I know. I'm going to risk thinking that since Microsoft are contributing to the chrome codebase, and the default supposition is surely that Microsoft wouldn't want any advancement of any features that have desktop equivalence, maybe a diplomatic possibility exists for introducing the semantics and accessibility consistency we desperately need, by a little sacrifice. I would love to be able to hit a key combination inside a Azure page, that thanks to the aforementioned accessibility standards, had a (possibly expanded) command line consistent equivalent and, permitting a little browser specific markup, could, with a single further stroke, break the interaction out of the browser and into powershell. It seems to be the aim for Microsoft to be the universal developer interface for all. This is entirely consistent with the same idea. The number of developers is only likely to be increasingly close to the whole user base of PCs, as voice and other interfaces develop. Get VSC for VB6 on Android and the job's done, in one short generation ...
The API is probably your only chance..
A dashboard interface for messages/notifications in VSTS would be helpful.
Aside, switching to a GraphQL type interface with something that updates your UI consistently would be nice. Some screens update fine, others just get lost/disconnected and you don't even know where you are at.
Overall, I haven't experienced the agent issues in TFA, but I've had plenty of other issues and continue to use the beast that I know. I do think the issue tracking is decent and the integration is as nice as other products I've tried imho. Sometimes simpler is easier though.
I feel like I must shoulder some of the responsibility here for not being louder about these issues. But must say, I'm disappointed to hear you're "surprised" about the UX issues. I've been telling your folks the UX is dreadful (e.g. as far back as pre-launch) and kept hearing back "we know, we're fixing it". I'll start formalizing the feedback and push it through the pipes, stay tuned. I'm also local (Bellevue), would love to come in and try to pipeline our relatively simple oss .net/wpf/uwp app. I suspect it'll be an eye opener for the both of us.
Some examples:
* You can't build a pipeline with a git repo. that contains submodules
* Found it impossible to edit the PATH for some custom tooling
* The New Pipeline experience just doesn't make a lot of sense, new users clicking around will eventually end up at the wrong Docs.
I believe their feedback mechanisms are placebo, pacifiers for loud people. Microsoft pesters you to give feedback and works hard to ignore them. And then goes ahead with what they initially planned. At the end, they'll announce - "We had delivered what you wanted. Based on feedback..."
And then you start wondering if you're in the minority and other people are simply crazy until you see multiple complaints about the same things.
Their music app, Windows phone, Windows 10, Skype and Edge are where I experienced this. In all cases, user experience kept getting worse with each "IMPROVEMENT."
The only time Microsoft listens is when people ignore official feedback channels and there are massive outcries on many fronts. Even then, Microsoft often retreats only temporary.
You should be able to recurse submodules in a pipeline. In the visual designer, select "Checkout Submodules" in the pipeline's get sources step. If you use YAML, set "submodules: true" or "submodules: recursive" in the checkout keyword.
You should also be able to specify environment variables (including PATH) with the env keyword. But do reach out and we can get to the bottom of this.
I kind of wish $bash (git for window's bash version) was available as a shell option as that might make things easier as well.
Another issue is searching is painful... "TFS" vs "VSTS" vs "Azure DevOps" and half the time, the results are dated.
Several products aimed at tech people seem to make this mistake. We are power users, we don't want stuff hidden away. Stop having UX that's designed as if we were trying to browse twitter.
For example, when you go to do a release and you of course want to do release notes, previously you could copy and paste the related issues output into any document.
Now its been CSSified to be big, bubbly, and utterly useless to do anything but look at, and there is no print or export feature.
I'm more used to the old UI but some features are not easily discoverable. I wrote some posts on how to build branches and pull requests as it's not obvious: https://unop.uk/build-and-release-all-pull-request-merge-res...
I'm on the "new" UI and I'm not seeing it. You've just re-skinned the same buggy and broken cruft that was there before.
It reminds me of the Visual Studio 2019 blog posts about new icons, themes, and fonts as their big new features while ignoring the massive performance problems people have been asking about for years.
DevOps with or without the new branding/UI is the same poorly designed product that doesn't know what it wants to be and is super un-polished. It would be better if you made it really good for one specific user scenario rather than poor for dozens.
(It's not just you. The "why did they spend all that effort to change things that didn't need to be changed" question has come up for me countless times upon seeing redesigns of other products.)
As someone who's used it through VSO/VSTS/ADO days it's got foibles but yes, it's improving continuously.
[1] https://docs.microsoft.com/en-us/azure/devops/release-notes/
I’d also add that I use 2017 and 2019 daily, and build by build, I see massive improvements for performance.
I've also had very odd issues where agent pools will have an idle worker but the pipeline will sit saying there are no agents, waiting. Eventually it goes, but... It's just a bad impression when you're showing it and it's not consistent.
Also, the UI. Tis horrible. Take a nod from AWS guys and get rid of the widgety attempt at entertaining some exec who doesn't use Azure. AWS interfaces load consistently for the most part and do a much better job in being not annoying, comparatively.
Another bad user experience is if you write a pipeline, by hand, without the GUI and try to run your pipeline - well, good luck. Authorization for certain components of your pipeline will fail, even if they follow the pipeline documentation to the T. Why? Because the pipeline web GUI workflow does linking of authorization between components on first run and if you only kick your build off using, say a code check-in, the pipeline will just fail. And it will give you a non-descript authorization error until you pull all your hair out and randomly do a commit from the Repository GUI of which it will pop up a message asking you to authorize the thing that had been failing for the last hour. Either document this better or give users a descript error of the actual problem. This, again, is oft where I find Azure lacking.
The UI keeps changing, but doesn't fix underlying bugs that have been around for ages.
Multiple issues where the builds don't actually update in the browser when they go through the steps (f5 only fixes), the builds show out of order in the UI, and many many other wonky things.
Also releases aren't YAML yet. Builds are, and are a good feature.
Azure Devops is almost perfect, and with a bit of TLC would be excellent.
I know it's probably getting translated to yaml from the old way which was a bunch of forms with build options, but we had some trouble with this. At the time, yaml releases were also still in development/preview mode.
One thing I'm going to miss is being able to discover things through the build wizard. Now it's just a blank yaml file and I feel that hurts the initial impression. I'd really like an option to build yaml files using the old way, but still get the yaml output for version control, that way I have all of the options and their potential values immediately available and I don't have to memorize and type everything.
There's a bunch I like about Azure DevOps though. Hopefully I haven't missed something really obvious.
Great example here: https://visualstudio.uservoice.com/forums/330519-team-servic...
It looks like you did something that users didn't actually ask for in 2017, closed this, and then have been ignoring 1.5 years worth of comments being told what users actually want. From our end it's simple - the embedded burndown chart on the scrum board should be able to show story points, not just hours (like with any other scrum board tool). I shudder to think how many engineering hours went into solving the wrong thing here...
So sure, I'm missing caching that I've had on Travis. But honestly, it builds reliably fast anyway. On Travis, I always needed to wait a LONG time to get macOS agents, on Appveyor, we were limited to 2 parallel builds. Here, everything starts all at once, completes under 10 minutes (there's quite a lot to build and test).
If I could, I would love to have an sccache ( https://github.com/mozilla/sccache ) compatible distributed free caching solution for my C++ projects. That'd be a killer feature!
Haven’t looked to deeply what devops has to offer yet, but things I’d be looking for before migrating.
- I want to be able to develop, run and debug most, or all, of the steps in the graph on my development machine without roundtripping through long chains of triggers and pushing to code repos and pull requests
- I want to run as much as possible in parallel, preferably agnostic of wether it’s on the same machine or several
- I need to build and assemble many components from same repository (monorepo setup)
- only impacted components should build
- Sub-components may depend on different build tools (service fabric services with both webpack and msbuild based packages)
- Some steps may depend on configurations and secrets with a different lifecycle than the code branch being built
I imagine something clever could be built on docker to support both running things locally as well as customize build nodes.
I don’t care to much for scripting in yaml. In fact I spent the day converting some yaml (and cake) scripting to Invoke-Build. Invoke-Build is quite nice actually, the only thing I miss is the option to orchestrate tasks in parallel onto several build nodes. That plus something like ncrunch build/test engine for csharp projects
(really the entire vs/msbuild things needs to be replace with something truly distributed, incremental and repeatable)
Builds are 100% stateless within the CI cluster and rely on tracking/publishing just about anything you can imagine or plug as a resource; out-of-the-box inclusions being docker images (registry), S3 (e.g. minio, AWS) and git.
It's a bit strange to reason about in the beginning but I have yet to try anything that comes remotely close to Concourse when it comes to orchestrating across multiple sources of input/output and non-linear pipelines.
So I think that I'm not understanding what you're creating, or where. Can you drop me an email with a screenshot? It's my HN username @microsoft.com. Thanks!
I have a 4K laptop, evan on high res, the lack of horizontal scrolling in Azure portal and other Azure tools is a dailly irritant.
Stop trying to badly emulate Android phone look & feel ... for desktop usage !!!
We are professionals, we can handle scrollbars without screwing up.
However, this confidence was seriously undermined when I noticed recently that my GitHub pull request validation build stopped working when I closed the DevOps tab.
Yes, seriously. I have to keep the DevOps tab open for my builds to actually trigger. My mind boggles at how such a design could even be possible.
The UI has so far changed five times in the past three months. It is insane. The current iteration is at least fine. Before it was a big blob of round circle that showed that the steps that succeeded succeeded. And then a big text saying it failed. It was worthless.
Maybe you should start investing smartly.
beetlejuice,beetlejuice,BEETLEJUICE!
- GitHub for source control
- AppVeyor for CI
- BlazeMeter for Load Testing
- Google Cloud for Cloud Hosting
- Slack for Team Chat
- MyGet for private NuGet feeds
- Trello for task management
- ELK cloud for log management
Yes it is many different services, but as a small team I like that we can pick the best tool for the job instead of trying to find a hammer for all nails which doesn't exist.
It also makes us less worried about changes, because we are not invested in any vendor so much that if they do something which makes our lives too difficult that we couldn't easily move on elsewhere.
Also it forces us to design our software in such a way that we don't lock ourselves into a specific vendor. For example we would have all our builds scripted instead of configured via some GUIs with the configuration stored on some vendor's cloud. The build script runs from anywhere. We can run our builds on Azure DevOps, TravisCI, CircleCi or anywhere. We just prefer to use AppVeyor because it's the smoothest CI server IMHO, but the transition elsewhere is just a matter of invoking a single build script.
I genuinely also prefer to work with a product which is doing one thing really good instead of having one product which does everything mediocre. From an operations POV it is just as simple as having everything in one place. We have one browser tab open for each service and it makes us actually a lot more productive. I am always only 1-2 clicks max away from what I need.
Teams/Companies who choose an all-in-one solution (e.g. everything in Azure + Azure DevOps) don't care about the productivity of their teams IMHO. They care more about the one person who has to do the accounting for their subscriptions or something like that :/
I have not had the same issues with build machines disappearing into the ether (we use Windows hosts however) but definitely agree that the (lack of) caching is maddening. Our apps are not huge but the NuGet & npm restore times now account for 25-30% of our build time.
I haven't had a build agent disappear, but lack of build caching is very frustrating. The hosted build agents are slow to begin with, let alone needing 5 minutes for every build to restore packages.
AFAIK there's no way to "only run changed tests" too.
EDIT: Forgot to add that, despite the above, I'm pretty happy with it. The whole VSTS experience (Wiki/code/PRs/etc) is pretty great, it's nice to have everything in a single place.
https://docs.microsoft.com/en-us/azure/devops/pipelines/test...
We tried to also use Azure Boards (Work Items, Boards, Backlogs, etc). Ouch. It is a complete UI mess of disjointed ideas. Instead of implementing one thing well, they implemented two dozen things terribly.
We actually migrated from Azure Boards to Office 365's Planner (KanBan) just because the UI wasn't so horrible. It is simple and gets out of your way. In fact O365's Planner, Teams, and OneNote Online products might be superior to anything Azure DevOps has produced.
The only nice thing I have to say about Azure DevOps Boards is that it integrates into Visual Studio, but frankly even that is barely passable. I'm predicting they scrap DevOps completely and just use GitHub as a clean slate.
May want to checkout GitLab as an option as well.
A little rant:
To the cloud infrastructure of MS: I don't know if anyone uses it for anything serious and doesn't complain about severe performance issues not only related to builds, but everything. From administrative sites to hosted environments. I read a lot of excuses mainly saying I should adjust my expectations regarding speed for cloud solutions... Problem is that there are other providers that don't have these issues.
Seriously, you need only to use office 365 if you want to get an impression of bad performance. Because everything is slow.
This is from a location in Europe, but I really doubt there are significantly more performant locations.
Microsoft are always last place in terms of performance and reliability.
For Kubernetes I actually keep track of exactly how bad they are with an automated testing tool.
Results of the last test are here: https://kubedex.com/is-azure-kubernetes-aks-any-less-terribl...
I've complained about UX before in the context of the Azure portal being, in my opinion, terrible. But that's subjective and I respect that others may like it, although not sure how they do.
Also, I've found in terms of UX GCP > Azure > AWS personally.
The client I was working with is very big and notable, so they had lots of help from Microsoft, and even so, there were so many problems with the service. I'd hate to see how smaller customers are treated.
OP's story about build agents turning unresponsive makes me think of this. MS is the best cloud vendor to deal as a consulting partner, yet the overall suffering caused by Azure almost makes me feel like it is not worth it.
Now, can we talk about the real WTF, which is the name. "Azure Devops", really? Two extremely potent keywords each on their own? You didn't like people being able to google answers to their questions about your product?
Another nicety would be to pay StackExchange to add an Azure DevOps subsite, assuming there isn't already one.
AWS slowly crept in because there was no entry barrier, no magic Harry Potter spells to get anything working and it actually worked and stayed working.
Now I went back to azure to see what has changed Q3 last year and it was the same “eating sawdust” experience. Literally everything is clunky.
When it comes to these providers, time is money, possibly more so than worrying about the instances. Azure costs a lot more in human attention.
AWS is not perfect though. CloudFormation quite frankly sucks balls but there’s enough interest and so little friction that there are lots of third party things that work well with it.
Azure, not so much. It’s a leper colony.
Even if I run on some other cloud than Azure, I still need the software. I could even run Azure DevOps on Amazon’s cloud I guess (If I choose the self hosted option) but the UX complaints would apply there as well...
Additionally, we are a non-profit, and MS works with us in a moderately generous way with pricing, whereas AWS wouldn't.
2. I think Azure is better overall than AWS. It's way easier to grasp the different features, while I haven't used Azure Devops, I've had little issues with the rest.
We're open to a better name for Auto DevOps https://docs.gitlab.com/ee/topics/autodevops/ but we haven't been able to find one. Suggestions are very welcome.
BTW We're planning to add per per minute for the CI runners and add shared MacOS and Windows runners so people that don't want to manage their own no longer need to.
My experience with using DevOps nee VSTS was mostly satisfying. For various reasons unrelated to the post, my team ended up installing the build agent on our own bare metal hardware, and other than needing to restart the agent, I didn't have any mission critical issues. Totally agree with all the UI complaints.
The CI component, Azure Pipelines, is the only hosted CI tool that supports all major platforms, at a minimum Windows/macOS/Linux.
The value of essentially fully automating out the role of DevOps engineers is a long time coming, and I used to do that sort of work but saw the writing on the wall with things like Azure DevOps. I think a lot of existing DevOps roles will transition over time to 'Cloud Engineer', specializing in setting up and maintaining hosted CI/CD platforms. There will also be a lot less of them as a result. It's becoming a commodity, MS is just one player there. I love what they're doing and eagerly embrace it, Microsoft has been knocking it out of the park. It works well for us.
I have seen a lot of these same problems noted in this link as well, just didn't see to prevent from getting our work done as much. So YMMV depending on your product/devops use case.
Personally have been using their APIs with some custom CLI scripts to get around some of these issues.
While I appreciate the work they put into DevOps, it still feels like the same person that designed the Azure Portal UI has had a say in the UI redesign of TFS into DevOps and that must either mean that a) Microsoft never realized what a horrible UX the Azure portal is, or b) they do realize but they are too invested in this new "design language" to do anything, so instead DevOps adopted the same style as Azure.
I'm in the process of migrating TO DevOps (on prem) so obviously I have a keen interest in the direction its taking. Build pipelines being declarative, navigation being clean up etc. are fantastic improvements. But it still feels clunky to use, especially around the backlogs/boards.
[1] https://docs.microsoft.com/azure/devops/pipelines/process/co...
[2] https://docs.microsoft.com/azure/devops/pipelines/process/se...
Sounds a lot like some process on the build server is leaking memory. We have a couple of older services that have this problem, and if we don't catch it early enough the server will become unresponsive and unable to even handle ssh connections.
That said I agree with a lot of the criticisms noted here (no retrigger on web hooks to update status, complicated web ui, log ux, lack build cache). It is possible to bring your own cache implementation, I just stuffed/restored things into object storage, also filed an issue for build cache https://github.com/Microsoft/azure-pipelines-tasks/issues/91... noting several other closed issues for the same.
It does feel like the platform is moving forward. All that said there's a lot of complexity and feature set in azure DevOps and pipelines that I haven't explored and try to ignore in the ui. At the moment its the only cloud provider build solution that actually even attempts to work well for opensource projects afaics.
One thing that needs to be improved so badly is their Test Plans section, not so fan for their way to setup agents for Load Testing.
I have small complaints here and there but I couldn't find another build service that has Windows and OS X agents for anywhere near what Microsoft charges.
Since this morning, we are stuck with an error message "TF400893: Unable to contact the server. This is most likely caused by a network error. Please check your connection and try again.". This has cause a couple of release pipelines to be get stuck, with cancel option not working either. The paid "parallel jobs" are being consume in this stuck pipeline and no other project from my organisation can be built or deployed because of it.
Pretty much regretting having spent so much time configuring the pipelines. We might switch it doesn't get resolved soon or if it happens even one more time.
Yes, the mobile page could be optimized (but this is not a deal breaker) and yes some pages do not subscribe for notifications, but usually all these things are not in the hot path.
TL;DR: Lots of improvements in the recent year and many things to come - it may not be perfect but it certainly is usable and provides value to our team.
We use the other stuff more than the CI, but will probably begin using it for some more smaller projects soon. It'll be in the same place which is great and no need to tinker with servers which is awesome.
> .NET MVC Web Application Build and Deploy Using FTP > .NET MVC Web Application Deploy in Cloud (Both Azure and AWS) > .NET MVC Web Application Build on Linux Visual Studio Agent Create Docker Image and Published
I didn't face any errors from devops, my experience overall good with DevOps. but i agree that it's not user friendly in term of UI and other workflow.
Microsoft also has operating expenses. They need to bring home some income to keep the lights on.
But Azure Portal UX is another story, the Blade UI is one of the worst things I've ever used, I hate scrolling horizontally, I can't open things in a new tab, it's really frustrating.
Any big reasons DevOps is something to consider, or is it mostly ideal for new projects?
The single issue I agree with is the lack of caching - installing NPM packages can take up to 15 minutes in the worst case. MS have known about this for years, so I honestly don't understand why they haven't done something about it.
I also think the hosted build machines are underpowered, despite their claims of Xeon CPUs - build on even an Azure B1ms VM (a low-powered, burstable SKU) are up to 3 times faster. In fact this is how we solve the NPM caching and performance issues - by running the VSTS Build Agent (it's OSS!) on relatively cheap Azure VMs.
Aside from these niggles though, I actually think the Azure DevOps product is fantastic.
1.) Make sure you are using `npm ci` instead of `npm install`. The `ci` install command is often much faster and skips a lot of stuff that `install` does. It's also better at sticking purely to your package-lock.json versions. I suggested on UserVoice at one point that Azure Pipelines should switch to `npm ci` as the npm task default. `npm ci` is a relatively recent npm command and I don't everyone has caught on to it yet.
2.) You can use Azure Artifacts for NPM packages. It acts as an alternate NPM feed and caches packages to your Azure DevOps account. Artifacts only supports NPM and NuGet, so it isn't a general caching solution, but in this specific case it helps. I've noticed the builds I have using Artifacts are a bit faster than going straight to NPM. (I started using Artifacts for private NPM packages, so I'm not using it in every build yet.) The current biggest caveat to using Artifacts as your main NPM feed for a project is that it doesn't support NPM audits currently. (I also made sure to report that on UserVoice.)
I can't find an actual issue and again as I mentioned in the post, replacing my AWS instance with a new AWS instance also eventually hit the same issue. Maybe it's some perfect storm of problems but that is just more speculation.
Apart from one minor issue with a bug in node.js timing when running the build server on burstable VMs (which is probably more to do with AWS setup than node), I've not had any issues that I can think of.
I don't have the problem with cache though, as I always run my own build servers/agents.
The UI has changed a few times over the past year, mostly around the rebrand from VSTS to Azure DevOps. But once learned, it's pretty straightforward and I'm sure I've barely touched the surface of its capabilities.
I, for one, am very impressed.
EDIT: oh, one improvement I can think of, would be to accept web-hooks rather than polling Bitbucket. But honestly, that's only a delay of a minute or two, so it's not a big deal. You could also easily enough create your own serverless webhook listener that triggers the build, so it's a bit of a non-issue, but I was surprised at the lack.
The New UI is a clear step backward...and the Github integration leaves a lot to be desired. If you have Github Enterprise...why use Azure Repos...why are MSFT teams moving from Azure Repos to Github...and others moving from Github to Azure Repos. The SCM platform and strategy obviously hasn't settled yet...so Gitlab is just as viable an offering that's growing more compelling by the day.
The Azure DevOps hosted build system hasn't been viable in my opinion since it launched, so on-prem agents are a much better option. Take a look at CloudBees offerings, they set the poll position.
The key thing that continues to be a pain point is the reporting and project organization data structure limitations with Azure DevOps: is it 1 TPC per business unit, 1 TPC for the whole company, 1 team project for the whole company. What happens when you have 2...how do you consolidate...what's the path forward if you have to move things around. The answer for the last 8 years has been silence.
The request for a clean solution has been asked for since 2011. Yes 2011, look at the link below. 8 long years...and basically silence...and "it's a hard problem"...and multiple updates saying "We will provide an update once we start planning for the second half of 20XX."
https://visualstudio.uservoice.com/forums/330519-azure-devop...
Eventually, the promises just ring hollow, the proof is in shipped code, and in the meantime other platforms and competitors stacks keep building out better enterprise stories for migrating from Azure DevOps (here's to look at Github, Gitlab, Atlassian's platforms, Jenkins/CircleCI, TeamCity, Artifactory, Xebia Labs etc).
Honestly, the world is a MUCH more competitive place with lots of best in class point solutions that aren't that hard to stitch together and get better performance/features.
Hope springs eternal...that with the new vertical offerings the DevOps team can deliver rapid value...but 8 years is a long time to only hear crickets.
If you're starting out and you need a one-size fits all solution and you're a SMB company with two doze devs...go hog wild with Azure DevOps...that's not to say it doesn't work for large companies like Shell and Microsoft itself...but talk with teams about the limitations they run into and make your own decision on your DevOps transformation adventure...oh and never create more than 1 team project in Azure Devops...seriously...if you do, game over.
Probably the biggest problem I've had was that build agents are stateful. The agent checksout a repository when the build starts, and removes the directory when it's done. It doesn't however clean up other changes made on the system, such as cache directories for tooling and other data you might move out of your build directory. I believe that builds must be reproducible, and therefore must start with a clean and consistent environment. You don't want builds to fail down the line because a precious job broke the agents environment, or even worse, having jobs succeed only on the agent because some dependent files are left on the agent. Other CI solutions use Docker containers for this to build in, which is great, as they are immutable (or stateless, rather) by default. This is in my opinion the single worst 'feature', making these CI builds distrustful.
Secondly, automating these agents is hard. If you want to automatically scale these self-hosted build agents horizontally based on demand, good luck! When starting an agent for the first time, you've to go through some cumbersome configuration wizard to set it up. There is no simple static configuration file available to use instead, or at least, I wasn't able to find it at the time. To mitigate the previous problem with stateful agents I created a Docker container having an agent in it, which was in fact quite difficult to properly achieve. I automatically restarted these agent containers multiple times a day in order to clean-up any garbage that was unintentionally left by a job. This is a horrible hack to slightly improve their (in my opinion) broken system, but it had the bonus of making these agents more easily scalable.
Another thing is the build configuration. I prefer having a configuration in a simple YAML file in the same repository, to have it as _close_ to the code as possible. Although Microsoft does provide such functionality it doesn't seem to be common to do it this way, and if I remember correctly it's much more difficult to properly set-up and use than for example, a Travis CI configuration. They do provide a graphical interface to configure these build steps though, which is useful when getting started with the system so you can get a feel of what is possible. It provides some graphical blocks/steps to achieve various thighs, such as doing an FTP upload or running a CLI command, with all kinds of weird properties. Maybe that's because Microsofts things are generally less scriptable. But yet again, I prefer a simple script in an in-repository configuration file, because it feels somewhat constrained and limited.
Sadly, it doesn't stop here:
- sometimes jobs just didn't want to stop, and no, bashing the abort button a million times didn't fix things. In some situations it took an hour before jobs were properly killed.
- the interface (VisualStudo TeamServices) is horrible to navigate, have had to wild-click many times to find what I need. I'm sure this will improve once you use it for a longer period though.
- and there are many other inconsistencies and little problems, such as build logs loading half of the time.
Maybe I'm grumpy. Maybe I'm ranting about nothing. I don't know. Maybe things have changed, or will change, at least I hope it will. I didn't have a great time with their CI system, my experience was horrible as you can probably tell by now. It feels like an unfinished product, that was released to quickly just to satisfy customers, so they didn't have to switch to some other platform not owned by Microsoft for their CI needs. What amazes me is that there are quite a few free alternatives available which are much simpler and more elegant, such as GitLab, Travis and Circle CI. They are easier to navigate, are usually quite quick, don't have the issues I've (and the article have) described. And greatest of all, you can get it up and running in minutes without having to worry about it afterwards.
I could find no evidence, looks like the guy is an android developer of some sort.
IMHO, Microsoft has been getting a lot of things right lately. Keep up the good work.
